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Objective 

We performed an evaluation of the F-35 Light- 
ning II Program [F-35 Program] by conducting 
a series of quality assurance assessments of 
the Joint Program Office, prime contractor, and 
major subcontractors. We assessed conformity 
to the contractually required Aerospace 
Standard (AS]9100, "Quality Management 
Systems - Requirements for Aviation, Space 
and Defense Organizations," contractual 
quality assurance clauses, and internal quality 
assurance processes and procedures for the 
foUowing six contractors: 

• Lockheed Martin Aeronautics Company, 
Fort Worth, Texas [Prime Contractor and 
Aircraft Integrator]; 

• Northrop Grumman Aerospace Systems, 
El Segundo and Palmdale, California 
[Center Fuselage Integrator]; 

• BAE Systems, Samlesbury United King- 
dom [Aft Fuselage Integrator]; 

• L-3 Display Systems, Alpharetta, Georgia 
[Panoramic Cockpit Display System]; 

• Honeywell Aerospace, Yeovil, United 
Kingdom [On-Board Oxygen Generation 
System]; and 

• United Technologies Corporation, Aero- 
space Systems, Fort Worth, Texas, 
and Independence, Ohio [Landing 
Gear System]. 



Findings 



The F-35 Program did not sufficiently implement or flow down 
technical and quality management system requirements to prevent 
the fielding of nonconforming hardware and software. This could 
adversely affect aircraft performance, reliability maintainability and 
ultimately program cost. Lockheed Martin Aeronautics Company 
[Lockheed Martin] and its subcontractors did not foUow disciplined 
AS9100 Quality Management System practices, as evidenced by 
363 findings, which contained 719 issues. 

The Joint Program Office did not: 

• Ensure that Lockheed Martin and its subcontractors 
were applying rigor to design, manufacturing, and quality 
assurance processes. 

• Flow down critical safety item requirements. 

• Ensure that Lockheed Martin flowed down quality assurance 
and technical requirements to subcontractors. 

• Establish an effective quality assurance organization. 

• Ensure that the Defense Contract Management Agency 
perform adequate quality assurance oversight. 

In addition, the Defense Contract Management Agency did not: 

• Sufficiently perform Government quality assurance oversight 
of F-35 contractors. 



Recommendations 

The Joint Program Office should: 

• Ensure compliance with AS9100 throughout the F-35 
supply chain. 



Visit us on the web at www.dodig.mil 
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Recommendations Continued 

• Ensure that Lockheed Martin approves all design and 
material review board changes and variances with 
Government concurrence. 

• Perform process proofing of all critical processes to 
include first article inspections. 

• Modify its contracts to include a quality escape* 
clause to ensure the Government does not pay for 
nonconforming product. 

• Assess the impacts and risks to all delivered aircraft for 
all findings. 

• Implement an aviation critical safety item program that 
meets the requirements of Public Law and DoD policy, 
which would include flow down of requirements for a 
critical safety item program to Lockheed Martin and 
its subcontractors. 

• Assess the impacts and risks to all delivered aircraft for 
critical safety item deficiencies. 

• Perform technical and quality assurance require- 
ment flow down and verification throughout the F-35 
supply chain. 

• Establish an independent quality assurance organiza- 
tion, which has the authority and resources to enforce 
the AS9100 Standard and F-35 product quality. 

• Revise the Defense Contract Management Agency 
memorandum of agreement to include explicit quality 
assurance oversight requirements. 



A quality escape is nonconforming material that has entered the product, 
supply chain, or proceeded beyond the acceptance process. 



• Ensure that Defense Contract Management Agency is 
performing quality assurance oversight commensurate 
with product criticality. 

The Defense Contract Management Agency should: 

• Provide a comprehensive quality assurance oversight 
plan for Joint Program Office approval to be included in 
the memorandum of agreement. 

• Audit the execution of the quality assurance oversight 
plan throughout the F-35 supply chain. 

Management Comments and 
Our Response 

On August 23, 2013, the Joint Program Office and the 
Defense Contract Management Agency responded to the 
findings and recommendations in the report. The Joint 
Program Office agreed with eight recommendations, 
partially agreed with two, and disagreed with one. The Joint 
Program Office stated that it does not have the resources 
to perform process proofing of all critical processes nor 
has the responsibility or resources to perform requirement 
flow down verification throughout the F-35 supply chain. 
However, we disagree because it is the Joint Program Office's 
responsibility to ensure contractual compliance to prevent 
nonconformances. It is also the responsibility of the Joint 
Program Office to update the contract if the requirements 
are deficiënt. 

It was also our recommendation that Joint Program Office 
establish an independent quality assurance organization 
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Management Comments and Our Response Continued 

reporting to the Program Manager The Joint Program Office 
disagreed stating that the Defense Contract Management 
Agency performs the role of the independent quahty 
assurance organization for the F-35. We disagree because 
the Defense Contract Management Agency is not 
accountable for program quality assurance goals. An 
independent quality assurance organization reporting 
directly to the Program Manager would ensure that 
performance and reliability objectives are met. 

The Defense Contract Management Agency agreed with one 
recommendation and partially agreed with the second. The 



Defense Contract Management Agency stated that it would 
update the memorandum of agreement between the Defense 
Contract Management Agency and the Joint Program Office, 
regarding surveillance; however, we disagree and desire 
specifics on the level of oversight at contractor facilities. 

The following table identifies recommendations requiring 
an additional comment by the Joint Program Office and 
Defense Contract Management Agency. Please see the 
Overall Findings and Recommendations section in the 
report for details. 
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Recommendations Table 



Management 


Recommendations 
Requiring Comment 


No Additional 
Comments Required 


Joint Program Office 


A.3, B.2, C, D, and E.l.b 


A.1,A.2, A.4, A.5, B.1, E.l.a 


Defense Contract IVlanagement Agency 


E.2.a and b 





*Please provide comments by October 28, 2013. 
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MEMORANDUM FOR PROGRAM EXECUTIVE OFFICE JOINT STRIKE FIGHTER 

DIRECTOR, DEFENSE CONTRACT MANAGEMENT AGENCY 



SUBJECT: Quality Assurance Assessment of the F-35 Lightning II Program 
(Report No. DODIG-2013-140) 

The DoD Inspector General [IG] conducted a quaüty assurance assessment of the F-35 Lightning II 
aircraft procured from Lockheed Martin. We conducted the assessment at the Joint Program Office (JPO), 
onsite at Lockheed Martin, and at the major subcontractors during FYs 2012 and 2013. Our objective was 
to assess quaüty assurance conformity to regulatory and contractual requirements necessary for 
F-35 aircraft production. 

Our assessment determined that the F-35 JPO oversight of Lockheed Martin was inadequate and that the 
Defense Contract Management Agency (DCMA) oversight of the contractors was ineffective. These issues 
may result in nonconforming hardware, less reüable aircraft, and increased cost. Throughout the assessment, 
we issued Notices of Concerns to the F-35 JPO to ensure timely corrective action of our findings. It is our 
understanding that the JPO has been implementing corrective actions and the DoD IG will perform future 
assessments of this critical program. 

The draft version of our report made recommendations to the F-35 JPO and DCMA to ensure compüance 
with quahty management standards throughout the F- 35 supply chain and to assess the impacts and risks 
to all deüvered aircraft. We considered management comments on the draft from JPO and DCMA. 
We request further comments from the JPO on Recommendations A.3, B.2, C, D, and E.l.b and from DCMA on 
Recommendations E.2.a and E.2.b. Further comments should be received by October 28, 2013. 

DoD Directive 7650.3 requires that recommendations be resolved promptly. If possible, send a .pdf file 
containing your comments to alois.dopita@dodig.mil . Copies of your comments must have the actual signature 
of the authorizing official for your organization. We are unable to accept the /Signed/ symbol in place of 
the actual signature. If you arrange to send classified comments electronically you must send them over the 
SECRET Internet Protocol Router Network (SIPRNET]. 

We appreciate the courtesies extended to our staff Please direct questions to Mn Al Dopita at [703) 699-0220 or 
alois.dopita@dodig.mil . 




^ndolph R. Stone 
Deputy Inspector General 
Policy and Oversight 



cc: Under Secretary of Defense for Acquisition, Technology and Logistics 

Assistant Secretary of the Air Force [Financial Management and Comptroller) 

Naval Inspector General 

Auditor General, Department of the Navy 
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Objectives 

Our objective was to assess the F-35 Lightning II Program [F-35 Program], Joint 
Program Office (JPO), the prime contractor, and major subcontractors conformity 
to the contractually required Aerospace Standard [AS)9100, "QuaUty Management 
Systems - Requirements for Aviation, Space and Defense Organizations," contractual 
quahty assurance clauses, and internal quality assurance processes and procedures. 

Background 

The F-35 Program is a joint, multinational acquisition to develop and field an affordable, 
next-generation strike fighter aircraft for the Navy Air Force, Marine Corps, and eight 
international partners: the United Kingdom, Italy the Netherlands, Turkey, Canada, 
Australia, Denmark, and Norway. The F-35 has three variants. The Conventional 
Takeoff and Landing [CTOL], Short Takeoff and Vertical Landing [STOVL), and Carrier- 
Suitable Variant (CV). 

Lockheed Martin Aeronautics Company [Lockheed Martin) entered system development 
and demonstration in October 26, 2001. Lockheed Martin has two principal 
subcontractors/suppliers, Northrop Grumman Aerospace Systems [Northrop Grumman) 
and BAE Systems [BAE). Figure 1 shows the breakdown of the manufacturing for major 
assemblies between Lockheed Martin and the principal subcontractors. The program 
has about 1,300 other suppliers, with production occurring in 47 states and Puerto Rico. 
Additionally, production is occurring in more than 600 suppüers in 30 other countries. 

Figure 1. Manufacturing Breakdown of F-35 Major Assemblies 
Source: Image courtesy of JPO 



SYSTEMS 



• Aft Fuselage 

• CVWing 
Fold System 



• Center Fuselage 



• Forward Fuselage 
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According to the Government Accountability Office reports, JPO rebaselined the F-35 
Program in 2004 following weight and performance problems and rebasehned again in 
2007 because of additional cost growth and schedule delays. In March 2010, JPO declared 
that the program exceeded critical cost growth thresholds estabhshed by statute — a 
condition known as a Nunn-McCurdy breach. JPO continued extensive restructuring 
actions during 2011 and 2012 that added more funding, extended schedules, and further 
reduced aircraft procurement quantities in the near-term. The quantity of F-35 aircraft 
to be procured was not changed, but restructured plans deferred the procurement 
of 410 aircraft until 2017. In March 2012, JPO estabhshed a new acquisition program 
basehne for the F-35 program, which incorporated all program restructuring actions. The 
March 2012 basehne represented the fourth rebasehne since the program's inception. 

According to the latest acquisition strategy the F-35 Program is one of concurrent 
development, production, and sustainment with nine separate low-rate initial production 
[LRIP]^ dehveries. Each LRIP represents an increasing level of maturity as additional 
system capability is delivered. Each LRIP also represents its own contract that establishes 
the number of aircraft to be produced and its own acquisition approach. For LRIPs 
1 through 3, JPO's acquisition approach was cost-plus incentive/award/fixed fee, but 
for LRIPs 4 through 9 a fixed-price incentive [firm target] type contract is being used. 
The contract fee methodologies also differ for the various efforts on each contract with 
incentive fee for cost and schedule, award fee for timeliness and quality, and fixed fee for 
diminishing manufacturing sources. 

Quality Trend Data 

F-35 Program quality metric data show improvement in scrap, rework, and repair rates and 
in software and hardware quality action requests per aircraft. However, the Government 
incurred and will continue to incur a significant cost for these issues, either through the 
previous cost-plus incentive/award/fixed-fee contracts or via quality incentives on future 
fixed-price incentive-fee contracts. As of March 2013, metric data showed that there 
were, on average, 972 quality action requests per aircraft for LRIP 1, 987 for LRIP 2, 926 
for LRIP 3, and 859 for LRIP 4. Scrap, rework, and repair rates on average per aircraft 
were 13.82 percent for FY 2012 and 13.11 percent for FY 2013, thus showing only a 
moderate change towards reducing costs. Although it would be unrealistic to expect first 
production to be issue free, our contractor assessments indicate that greater emphasis 
on quality assurance, requirement flow down, and process disciphne is necessary if the 
Government is to attain lower program costs. 



^ LRIP as defined by the Defense Acquisition University is the "effort intended to result in completion of manufacturing 
development in order to ensure adequate and efficiënt manufacturing capability and to produce the minimum quantity 
necessary to provide production or production-representative articles for Initial Operational Test and Evaluation." 
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Assessment Criteria 

AS9100 Standard 

The F-35 LRIP contracts require that the contractor comply with AS9100. We performed 
our assessments to the AS9100C Standard because it was the current version at the time 
of the assessments, and any defense contractor obtaining certification would be required 
to meet that Standard. 

The AS9100C Standard breaks down quahty assurance requirements into five 
major clauses: 

• QuaUty Management System, 

• Management ResponsibiUty, 

• Resource Management, 

• Product Reaüzation, and 

• Measurement, Analysis, and Improvement. 

The Quahty Management System, Management ResponsibiUty, and Resource 
Management clauses require the organization to have a quahty assurance management 
organization that has all the resources and authority to affect the end-item quahty of 
the product. In addition, it requires the organization to have a quahty assurance manual 
and strict controls over all documentation, data, and procedures that affect the quality 
of the product. Product Realization covers the activities and processes necessary to 
bring a product into existence. 

Product realization is broken down further in AS9100 as foUows: 

• Planning of Product Realization, 

• Customer-Related Processes, 

• Design and Development, 

• Purchasing, 

• Production and Service Provision, and 

• Control of Monitoring and Measuring Equipment. 

Planning of Product Realization requires the organization to develop processes needed 
for design and development of product and includes elements such as procedures, 
quality assurance records, resource requirements, safety and reliability programs, and 
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inspection and test. Design and Development includes requirements that cover planning, 
inputs, outputs, review, verification, validation, and control of changes as related to 
design and development. Purchasing requires the organization to ensure that the 
purchased product conforms to specified purchase requirements and that all products 
purchased from suppliers are verified against purchase agreement requirements. The 
Production and Service Provision requires the organization to ensure that production is 
accomplished under controUed conditions using drawings and specifications, work 
instructions, production tools and software programs, monitoring and measuring 
equipment, and evidence that all production and inspection/verification operations have 
been completed as planned. 

Measurement, Analysis, and Improvement requires the organization to ensure the 
product continuously improves. The clause includes customer satisfaction, internal 
audit, monitoring and measuring processes and product, and control of nonconforming 
products to ensure continual improvement. 

Aviation Critical Safety Items Requirements 

In addition to AS9100, we assessed the JPO and the contractor's implementation of 
aviation critical safety items [CSIs) requirements. A CSI is a part, assembly, or support 
equipment whose failure could cause loss of life, permanent disability or major injury, 
loss of a system, or significant equipment damage. Special attention should be paid to 
CSIs to prevent the potential catastrophic or critical consequences of failure. CSIs require 
special handling, engineering, manufacturing, and inspection documentation to control 
and ensure safety of flight. 

Public Law 108-136, Section 802, "Quality control in procurement of aviation CSIs and 
related services," requires DoD to prescribe a quality control policy for the procurement 
of aviation CSIs. Joint Service CSI Instruction, "Management of Aviation Critical Safety 
Items," implements the DoD CSI program and establishes the policies, procedures, and 
responsibilities to manage CSI. The Joint Aeronautical Commanders Group [JACG], "JACG 
Aviation Critical Safety Item Handbook," implements the policies in the Joint Service CSI 
Instruction and describes the technical and quality assurance requirements for a Prime 
Contractor CSI program. 

Statutory and Regulatory Requirements 

Additionally, we assessed the compliance of applicable statutory and regulatory 
requirements to include requirements in the Federal Acquisition Regulation; 
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DoD Instruction 5000.02, "Operation of the Defense Acquisition System," Enclosure 4; and 
DoD 4140.1-R, " DoD Supply Chain Materiel Management Regulation," Section C3.6. 

Quality Assurance Assessment Process 

To evaluate the JPO's management of the F-35 quaüty assurance program, we performed 
a series of quaüty assurance assessments of JPO, prime contractor, and major 
subcontractors. We assessed conformity to the contractually required, AS9100, 
contractual quaüty assurance clauses, internal quaüty assurance processes, and 
procedures for the foüowing six contractors: 

• Lockheed Martin Aerospace Company Fort Worth, Texas [Prime Contractor 
and Aircraft Integrator]; 

• Northrop Grumman Aerospace Systems, El Segundo and Palmdale, Caüfornia 
[Center Fuselage Integrator]; 

• BAE, Samlesbury United Kingdom [Aft Fuselage Integrator]; 

• L-3 Display Systems [L-3], Alpharetta, Georgia [Panoramic Cockpit 
Display System]; 

• Honeywell Aerospace [Hone3wen], Yeovil, United Kingdom [On-Board Oxygen 
Generation System]; and 

• United Technologies Corporation, Aerospace Systems [UTAS], Fort Worth, 
Texas and Independence, Ohio [Landing Gear System]. 

We selected the contractors based on product criticality and risk. For each assessment, 
we established teams of engineering and subject matter experts who assessed to the 
AS9100C Quality Management System Standard. The subject matter expert teams 
consisted of 14 to 18 quality assurance engineers, trained and certified in AS9100, who 
had an average of 15 years of quality assurance audit experience. Additionally at the 
Lockheed Martin, Northrop Grumman, and BAE assessments, we included a team that 
evaluated the aviation CSI process. 

This assessment focused on quality management system compliance. We did not 
examine whether financial restitution was granted for nonconformances, variances, 
waivers, deviations, etc. 
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Notice of Concern 

At the conclusion of each contractor assessment, we issued a notice of concern [NOC] to 
the JPO to ensure timely corrective action of each finding. The NOCs included the detailed 
findings from that location to ensure prompt resolution. 

Classification and Categorization of Findings 

We wrote 363 findings that identified a total of 719 issues for the six^ contractor 
assessments performed. There were multiple issues identified in most of the findings 
with the majority of issues were violations of the AS9100C Quality Management 
System Standard. For each of the assessments, we classified the findings as major 
nonconformances, minor nonconformances, or opportunities for improvement [OFIs). 
Each finding received an additional technical review for accuracy and classification. 

As defined by the AS9101 Standard, a major nonconformance is a nonfulfillment of a 
requirement that is likely to result in the failure of the quality management system or 
reduce its ability to ensure controlled processes or compliant products/services. A 
minor nonconformance is a nonfulfillment of a requirement that is not likely to result in 
the failure of the quality management system or reduce its ability to ensure controlled 
processes or compliant products or services. An OFI is an industry best practice where 
a specific requirement does not exist. This report focuses on the major findings for each 
respective contractor. 

The table shows the breakdown of major, minor, and OFI findings for each of the 
site assessments. 



Table. Major /Minor /OFI Finding Totals at Each Site* 



Location 


Major 


Minor 


OFI 


Lockheed Martin 


28 


42 





Northrop Grumman 


23 


42 


1 


BAE 


36 


44 


2 


L-3 Communications 


25 


30 


1 


Honeywell 


18 


20 





UTAS 


17 


34 





Total 


147 


212 


4 



*Findings include those written against JPO and DCMA during the respective assessments. 



^ We conducted a pre-assessment visit at Honeywell, Phoenix, Arizona. Although the site was not selected for assessment, 
we wrote one finding against the Defense Contract Management Agency. 
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For each of the contractor assessments, we categorized the findings by the 
relevant AS9100C clause. Figure 2 provides the breakdown of these findings for the 
overall assessment 



Figure 2. Categorization of Findings 
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Contractor Assessments 

Lockheed Martin Aeronautics Company 
(Prime Contractor) 

[Fort Worth, Texas) 

Lockheed Martin is the prime contractor and lead integrator for the F-35 Program. 
Lockheed Martin manufactures the forward fuselage and wings and performs final 
integration and final aircraft verification at the Fort Worth, Texas, facility [Figure 3]. 

Our assessment of Lockheed Martin resulted in 70 findings that identified weaknesses 
in Lockheed Martin's implementation of an AS9100 Quaüty Management System. Many 
of the issues documented in the findings indicate additional F-35 Program risks that 
could impact cost, schedule, and performance. We wrote an additional 24 findings 
against Lockheed Martin stemming from our assessments at its subcontractor sites 
[7 at L-3 Communications, 5 at Northrop Grumman, 7 at BAE, and 5 at UTAS]. Figure 4 
provides the Lockheed Martin findings by AS9100 clause with the foUowing sections 
summarizing significant issues documented during the assessment. 




Figure 3. Lockheed Martin Assembly Line Source: Image courtesy of Locl<heed IVIartin 
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Figure 4. Lockheed Martin Findings 
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Documentation Requirements (4.2) 

We wrote several findings regarding documentation control. For example, the material 
data system that automatically records the cumulative product excursion time^ of 
composites prior to curing requires corrective action. When products required a 
secondary lay-up, the excursion time continued to accumulate in the material data 
system, even though the product cured. To preclude the appearance that the materiel 
had exceeded its useful üfe, operators were overriding the automated materiel data 
system with manual entries. We could not determine if these manual entries accurately 
reflected the actual excursion time of the material. The ability of operators to override 
the recorded excursion time data prevents the data record from reflecting whether 
the final product conformed to requirements. 



The excursion time reflects tiie amount of time available for ttie operator to use tlie material, typically for bonding agents. 
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Other findings documented similar record control deficiencies including maintenance of 
flight line security logs and unclear record retention requirements. In addition, several 
procedures, checklists, and program plans were in use prior to approval and formal 
release. Lockheed Martin personnel were using unapproved [without engineering and 
quality assurance approval] procedures for 18 months. The unapproved procedures 
were used to control the vacuüm hold-down fixtures for both the flexible overhead 
gantry machining and coordinate measurement machine. In addition, the manufacturing 
plans for the wing systems and wing structures were labeled as "uncontroUed/for 
reference only," although they were used by the planning department to develop the 
work instruction for aircraft assembly. Maintaining accurate product records and 
controUing process documentation is necessary to ensure that the product meets 
engineering and customer requirements. 

Human Resources (6.2) 

Competence, Training, and Awareness (6.2.2). We found Lockheed Martin's 
management had not updated the employee training requirement to reflect actual 
training needs. The Learning Management System included 80 employees with expired 
certifications such as ejection seat installation and removal, F-35 egress system safety 
and explosives care, handling, and storage. In addition, employee training plans 
identified training certifications that were not required for the jobs those employees 
were performing. In another example, 37 operators were working on flight hardware 
in a foreign object debris/damage [FOD]-control area without current certifications. 
A subsequent major finding, noted later in this report [see paragraph 7.5.5), documented 
that FOD was discovered in FOD-critical and FOD-control areas. Training ensures 
operators are cognizant of the latest industry standards and techniques; it also provides 
increased awareness. The lack of management attention to certification requirements of 
employees working in aircraft assembly areas places the delivered product at risk. 

Planning of Product Realization (7.1) 

We identified several major findings in Lockheed Martin's Planning of Product 
Realization. A major finding noted that Lockheed Martin's shop floor planning contained 
incorrect verification steps that resulted in verifications either not being completed or 
unnecessary. For example, the center wing mate assembly work instruction required 
torque verification for fasteners; however, fasteners involved in this operation did not 
require torqueing. In another instance, the wing systems installation work instructions 
required performing two verifications; one for verification of the electrical bond and 
the other for electrical resistance. A review of drawing requirements and discussions 
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with personnel noted that the operators were not required to perform the electrical 
resistance verification, even though it was later determined to be required. Finally, the 
wing system installation work instructions identified two different inspection criteria for 
electrical bond resistance inspection; however, it did not specify which one applied. 

Another major finding noted that manufacturing documentation did not identify an 
inspection requirement that would verify that sealing of fasteners on skin installations met 
dimensional requirements. Specifically the drawing specified minimum fillet dimensions 
for integral fuel tank sealing; however, the dimensions were only inspected visually and 
were not measured. Sealing adds weight to the aircraft and because measurements were 
not taken, it is unclear whether this would be accounted for in final system performance. 

Planning and procedures in several process areas lacked sufficiënt detail to provide 
adequate work instructions. For example, in the metal machining areas, the procedures 
did not identify details regarding part cleaning, setup, and mechanical clamping; and 
in the wing assembly area, the fillet sealing tools were not identified in the procedure, 
and personnel were using unspecified tools. Also, a specification for the integral fuel 
tank sealing had numerous errors in the graphics depicting dimensional identification 
along with missing, incomplete, or incorrect dimensional lines. In the wing box 
integration area of the facility, the planning did not detail numerous operations for 
skin panel cover closeout. This lack of precise and accurate work instructions could 
result in nonconformances later in processing or after fielding, thus causing schedule 
delays and additional program costs. 

A major finding noted that Lockheed Martin was not implementing its diminishing 
manufacturing sources and materiel shortages [DMSMS] process in accordance 
with internal procedures. For example, Lockheed Martin did not always include the 
requirement for the delivery of bills of materials in its supplier statement of work [SOW) 
and did not always upload subtier supplier bills of materials into its obsolescence- 
forecasting tooi. This precluded Lockheed Martin from proactively managing DMSMS. 
Cost metrics provided by Lockheed Martin indicate that the cost of DMSMS has been 
increasing. Proactive management of DMSMS is critical for containing program cost. 

Lockheed Martin had not assessed the use of lead-free electronic parts throughout the 
F-35 Program as required by its own corporate requirements, "Lockheed Martin 
Corporation Lead-Free Control Plan." Lockheed Martin did not flow down a lead-free 
control plan requirement to its suppliers. The use of lead-free electronic parts in critical 
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applications increases the risk to component service life, reliability, airworthiness, and 
safety because of tin whisker growth.'* 

A final major finding identified that Lockheed Martin did not always define the capabiUty 
confirmation criteria used for acceptance of aircraft. In some cases, confirmation criteria 
did not include clearly measurable and verifiable acceptance criteria, or were not traceable 
to source data necessary to confirm the required capabiUty. As a result, acceptance of 
F-35 aircraft depends on the judgment of individual subject matter experts without the 
use of clearly measurable and verifiable acceptance criteria to confirm the capability of 
the aircraft. Unclear capability confirmation criteria in the configuration and capability 
description documents can lead to inconsistencies or quality escapes^ during acceptance 
of LRIP aircraft. 

Configuration Management (7.1.3). Our assessment identified that Lockheed 
Martin neither adequately provided review or approved of engineering change 
submittals made by Lockheed Martin's critical suppliers. Specifically Lockheed Martin 
neither approved or disapproved all L-3 major changes nor provided concurrence in 
classification for all minor changes submitted by L-3 and UTAS. L-3 submitted 8 major 
engineering change proposals to Lockheed Martin since May 7, 2007, with no evidence 
of approval or disapproval, and 14 minor changes through engineering change notices 
during 2011, with no response for "concurrence in classification." This indicates a 
breakdown in a basic quality assurance process used to ensure that Lockheed Martin's 
systems engineering understand and agree or disagree with supplier product changes. 
Another major finding noted that Lockheed Martin did not require L-3 to perform 
configuration status accounting for LRIP 1 through 5 delivered aircraft. Failure to require 
suppliers to perform configuration status accounting of delivered hardware can result 
in the inability to provide hardware traceability in support of failure analysis, system 
upgrades, and supportability of the aircraft. Lockheed Martin's configuration management 
processes did not define configuration management board membership and authorities. 
In addition, the configuration management plan, imposed on BAE by Lockheed Martin, 
had invalid references and processes for BAE to foUow. 

Design and Development (7.3) 

Design and Development Planning (7.3.1). A major finding identified that Lockheed 
Martin was delivering F-35 aircraft that did not meet required interchangeability- 



" A phenomenon known as "tin whiskers" becomes a factor due to the absence of lead in tin-bearing solder and component 

finishes. Tin whiskers can cause electrical shorts resulting in system failures. 
^ A quality escape is nonconforming material that has entered the product, supply chain, or proceeded beyond the 

acceptance process. 
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replaceability contract requirements dating back to 2001. This deficiency was identified 
in May 2009 by the Defense Contract Management Agency [DCMA]. Lockheed Martin 
identified this as a high-risk item and developed a 35-step plan for mitigation; however, 
final mitigation is not expected to be complete until 2015. The interchangeability- 
replaceability plan was to validate through demonstration 277 interchangeability- 
replaceability components that were primarily comprised of aircraft skin assemblies. At 
the time of our assessment, 273 demonstrations were still required and it was estimated 
that 150 aircraft would be fielded before completing the plan. This will inevitably 
create fielded aircraft components that are not interchangeable or additional cost to the 
Government for bringing fielded aircraft up to specification. 

A software safety engineer was not assigned to the software development integrated 
product team [IPT], as required by the Software Development Plan, to ensure that 
software meets all safety requirements. Without adequate product evaluation of mission 
system software, Lockheed Martin cannot ensure aircraft safety requirements are met. 

Lockheed Martin's Mission Systems Software Development Plan did not address all the 
requirements of failure mode testing specified in the Air Systems Software Development 
Plan. Additionally, the Test Readiness Review for air system software block 2A did not 
contain exit criteria, as required by the Systems Engineering Plan. If requirements are 
not specified in the program plans, and technical reviews do not contain exit criteria, the 
aircraft design may not meet all program requirements. 

Lockheed Martin was not providing adequate management of the development efforts at 
critical suppliers. Multiple findings were written regarding Lockheed Martin's failure to 
flow down development planning requirements. During the UTAS assessment, we found 
that Lockheed Martin flowed down the F-35 Brake/Skid Control System performance- 
based specification with incomplete and conflicting specification requirements and did 
not flow down required safety and mission-critical function information. Additionally, 
Lockheed Martin provided incomplete, conflicting, and ambiguous system requirements 
in the Panoramic Cockpit Display performance specification to L-3. 

During the Northrop Grumman assessment, we identified that Lockheed Martin did not 
maintain mission systems requirements traceability to the software-level requirements. 
Specifically the requirements were not derived from or traceable to the allocated 
software requirements and the top-level system requirements. Untraceable requirements 
cannot be verified for impact on system performance. In addition, Lockheed Martin did 
not maintain the air system block plan to reflect current capabilities planned for the 
F-35 Program software blocks. The plan had not been updated since August 2008 and did 
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not reflect current block planning, which would define air system capabilities. As a result, 
there is no authoritative document defining current and planned software capabilities. 
Without a current air systems block plan, software program capabilities may not be 
implemented when required. 

During the BAE assessment, we identified that Lockheed Martin was not ensuring that BAE 
was working to the current revision of the software development plan for the F-35 mission 
systems software. Software products developed by BAE using outdated mission systems 
requirements may not contain current development processes and acceptance criteria. 
In addition, several of the F-35 subsystem, software development plans did not include 
software maintenance, even though the software was already in the maintenance phase. 

For some critical design reviews [CDRs), Lockheed Martin did not establish or require 
exit criteria. Additionally Lockheed Martin did not formally close all design review action 
items and did not disposition or approve the UTAS failure modes, effects, and criticality 
analysis [FMECA] reports for the main and nose landing gear. 

The lack of part interchangeability, insufficiënt requirement flow down, and open design 
review action items, as well as failing to evaluate and approve or disapprove engineering 
deliverables represent product instability and risk. 

Purchasing (7.4) 

Our assessment documented several findings citing inadequacies in Lockheed Martin's 
oversight of its suppliers and management of subcontractor deliverables. For example, 
a major finding noted that Lockheed Martin did not manage subcontractor deliverables 
in accordance with the approved process. Many of the deliverables identified in the 
subcontract SOW and associated subcontractor deliverable requirements lists had not 
been delivered or were delivered more than 2 years late without foUow-up by Lockheed 
Martin. Several of these deliverables; such as the software development plan, configuration 
management process plan, manufacturing plan, electronic bill of materials, advanced 
quality assurance plan, and acceptance test plan; still require concurrence or approval 
by Lockheed Martin. Additionally, the data management system for tracking the status of 
supplier deliverables contained inconsistent information with requirements contained in 
the subcontract deliverable requirements lists. 

Several other findings noted inconsistencies in the supplier management and control 
process. For example, Lockheed Martin did not always conduct supplier evaluations as 
required. Additionally, the evaluation process for software suppliers did not adequately 
identify requirements for supplier approval and control, such as a detailed supplier 
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approval process, detailed oversight strategy, supplier risk levels, and the periodicity for 
supplier selection audits or in-process supplier audits. These evaluations are a primary 
input into the suppHer rating program. In another finding, several suppüers were issued 
corrective action requests; however, these suppüers were dehnquent in responding to 
the requests and Lockheed Martin did not follow-up to determine the status. Lockheed 
Martin cannot ensure suppüers are meeting program requirements without a robust 
suppüer management system. 

Production and Service Provisions (7.5) 

Control of Production and Service Provision (7.5.1). Lockheed Martin was not 
foüowing estabüshed procedures to maintain production control. For example, several 
mechanics were violating procedures by not wearing gloves while installing the fasteners 
for the wing close-up operation. These particular fasteners have a chemical film dry lube 
coating on the threads and require gloves to avoid contamination during installation. In 
addition, an operator performing the nutplate push test for the CTOL wing was not pushing 
the required amount of time at each test point resulting in uncertainty that proper epoxy 
adhesion was obtained. 

Two findings documented a lack of adherence to procedures regarding rework and repair 
documentation. Lockheed Martin performed rework on the Rear Spar Brackets without 
issuing a quality assurance report. Another finding identified undocumented damage to 
the aluminum shims of an aircraft assembly. Assembly personnel notified the supervisor 
of the approximate 3x12 inch damaged area to the aluminum shims; however, they 
did not write a nonconformance report in compliance with Lockheed Martin process. 
Nonconforming product should be identified and dispositioned by the appropriate 
engineering expertise to ensure sufficiënt repair and eliminate future occurrences. As a 
result, a serious quality escape could occur, affecting reliability or safety of flight. 

Production Process Verification (7.5.1.1). Several First Article Inspection [FAI)'' 
findings were documented against Lockheed Martin. The purpose of FAI is to obtain a 
representative production sample and ensure that aü key design characteristics have 
been achieved, critical processes are controüed, and unit-to-unit repeatability is attained. 

A major finding documented several deficiencies with Lockheed Martin's application of 
the FAI process. Specifically, Lockheed Martin applied AS9102 guideline requirements 
inconsistently. AS9102 establishes the requirements for performing and documenting 

^ AS9102 defines FAI as, "A complete, independent, and documented physical and functional inspection process to verify 
that prescribed production methods have produced an acceptable item as specified by engineering drawings, planning, 
purchase order, engineering specifications, and/or other applicable design documents." 
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FAIs. Lockheed Martin imposed this Standard on all suppliers of F-35 critical assemblies; 
however, it did not require conformance to AS9102 on a product fabricated within its own 
facility. In addition, Lockheed Martin's FAI process focused primarily on a 100-percent 
inspection of products that met engineering requirements and did not include verification 
of process stabiHty and personnel proficiency. These tenets are required by AS9102, on 
which Lockheed Martin based its FAI plan. Finally, AS9102 does not exclude assembües 
from FAIs, but Lockheed Martin excluded FAIs on major assembües such as the wings, 
tail, forward, center, and aft fuselage sections of the aircraft. Many process changes 
at Lockheed Martin were occurring in the major assembly areas and involved moving 
tooüng and equipment to perform out-of-station work. According to AS9102, changes 
in manufacturing processes, location of manufacture, tooüng, or materials provide the 
rationale for performing or reperforming fuü or partial FAIs. OveraU, Lockheed Martin's 
FAI process was ineffective, as evidenced by the numerous planning, tooüng, and 
training deficiencies documented. 

We found that Lockheed Martin flowed down conflicting FAI requirements to BAE and 
Northrop Grumman. Specifically the contractually imposed AS9102 Standard requires 
that FAIs be performed on all components, including assemblies. However, Lockheed 
Martin's FAI plan exempts many of the major assemblies and subassemblies from FAIs. 
This conflict resulted in many items not receiving FAIs. Additionally at L-3, Lockheed 
Martin had approved variances for L-3 to ship display units and electronics units to 
Lockheed Martin without completing FAIs; another indicator of Lockheed Martin's 
disregard for the FAI process. 

Failure to adequately perform FAIs could impact the F-35 Program's ability to achieve 
process stability and successfuUy meet program global production rate goals. The lack 
of process stability will increase costs and schedule because of discrepant hardware and 
quality escapes. 

Control of Production Equipment, Tools, and Software Programs (7.5.1.3). The 

assessment identified a major deficiency with documenting the control of manufacturing 
tooüng used on the F-35 Program. Lockheed Martin personnel were using discrepant 
sealant mixing equipment for production. Without engineering approval or analysis, 
Lockheed Martin personnel were mixing the sealant [for the skin installation of the 
left aft wingbox] at twice the required time to compensate for the discrepant machine. 
Furthermore, the compensation time used was incorrect, thus resulting in possible over 
mixing of the sealant. Other findings noted that additional equipment was not maintained, 
including the vacuüm system equipment used for composite machining operations, 
another sealant mixer, and software for computer numerically controUed machines. 
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Preservation of Product (7.5.5). We documented major findings related to control 
of FOD and shelf-life material. For instance, controls were inadequate to prevent FOD 
from being introduced onto the production floon A DoD Office of Inspector General 
[OIG) walkthrough identified FOD within FOD-critical areas located in the final assembly 
production area and flight hne run stations. Examples include metal shavings found 
throughout the left air intake of the aircraft, leaking fluid from a bagged hydrauHc Hne, 
and a paper tag below the aircraft. FOD was also identified in FOD-control areas located 
in the electronic mate, alignment system, and final assembly production areas. Examples 
include metal shavings found in the cockpit of the aircraft; and metal fasteners, wood 
fibers, and miscellaneous tools and debris throughout the production area. Another 
major finding documented several incidents where wiring harness protective connector 
caps were missing. The caps are designed to prevent both FOD and electrostatic damage 
to aircraft components and circuitry during manufacturing. 

We found expired composite material and a process lay-up kit with improperly identified 
shelf life and excursion times. These items were not impounded to prevent unintended 
use. Failure to properly identify and maintain traceability of shelf-life material may lead 
to use of material that does not meet specifications, potentially resulting in degraded 
material performance and failure. 

Control of Nonconforming Product (8.3) 

Airframes measured using the laser alignment system routinely did not meet mate 
and alignment drawing requirements; however, Lockheed Martin did not identify that 
the airframes were discrepant. Numerous airframes for all three aircraft variants have 
been integrated into end-item flight hardware and processed through final assembly 
but Lockheed Martin did not identify and disposition the nonconformances. Two other 
findings identified three separate instances when "Open" nonconformances were not 
identified and subsequently removed in accordance with procedures. The effective 
identification, review, and disposition of nonconforming product is essential to prevent 
processing of defective hardware into the aircraft. 

Improvement (8.5) 

Corrective Action (8.5.2). We wrote several findings regarding Lockheed Martin's 
corrective action process. At the L-3 assessment, Lockheed Martin approved variances 
that waived performance requirements for the electronics and display units. The 
variances did not contain documented corrective actions that would prevent the need for 
repeated variance requests. There were 17 performance variances on the display unit 
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and 8 performance variances on the electronic unit. L-3 continues to produce and deliver 
electronics and display units with performance issues under these variances. 

At the Northrop Grumman assessment, the software tooi [Eagle Speed) that Lockheed 
Martin provided to Northrop Grumman did not include Identification and recording of 
software nonconformance root cause as required in the F-35 mission systems software 
development plan. This tooi tracks and manages mission systems software problem 
anomaly and software problem reports. Without identification of the root cause of 
software problems, Lockheed Martin cannot conduct effective corrective action to 
address software deficiencies. 

Joint Program Office Oversiglit of Locl<t)eecl IVIartin 

The CSI findings documented at Lockheed Martin, Northrop Grumman, and BAE Systems 
indicate that JPO had not established a CSI program for the F-35 Program. 

JPO had not flowed down a contractual requirement to Lockheed Martin [and 
subsequently its subtier suppüers] to implement a CSI management program in 
accordance with Public Law and DoD Policy. CSIs require special handling, engineering, 
manufacturing, and inspection documentation to control and ensure safety of flight. 
Without a CSI program, there is an increased safety risk to the aircraft and warfighten 

Another major finding noted that JPO had not funded Lockheed Martin to create a 
DMSMS process and had not developed a DMSMS Program Management Plan; causing a 
significant cost risk to the program. However, according JPO, JPO was in the process of 
funding a DMSMS program. 

We also identified that JPO did not establish definitive exit criteria for the block 
2A Air System Test Readiness Review, reducing the probability that the system is ready 
to proceed into formal test. JPO systems engineering and risk management plans 
require contractors to continuously assess the F-35 Program risks. However, JPO was not 
reviewing and documenting program risks during the Program Management Advisory 
Board meetings or the risk-level management reviews. Without active participation 
in systems engineering, risk management, and lifecycle logistics planning for the 
F-35 Program, JPO cannot ensure that all program requirements will be met. 

Locl<t)eecl IVIartin Assessment Summary 

Our assessment of Lockheed Martin's quality management system, processes and 
procedures indicated a lack of discipline in complying with AS9100 requirements. This 
will result in nonconforming hardware, less reliable aircraft, and increased cost. 
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Manufacturing planning and process qualification were not at the level commensurate 
with current and planned production levels. In general, documentation contained 
inaccurate and incorrect verification steps and lacked sufficiënt detail to provide 
adequate work instructions. This indicates that processes remain immature and that 
assembly instructions require immediate updating. 

Lockheed Martin's management of requirements; including those flowed down to 
suppliers in the areas of design, configuration management, software, and FAI; were not 
always clearly defined, approved, maintained, and verified for compliance. In addition, the 
requirements provided conflicting direction. Without clear and accurate requirements, 
Lockheed Martin cannot ensure that flight hardware meets F-35 Program requirements. 

Finally, Lockheed Martin was not committed to FOD control. Workers were in the 
assembly areas with expired FOD certification, and discipline was generally lacking in 
FOD-control and FOD-critical areas. The inspection team noted substantial FOD in 
and around the aircraft; these FOD deficiencies continued even after Lockheed Martin 
management shut down the line twice to correct noted deficiencies during the OIG 
assessment. An ineffective FOD program can result in damage to F-35 aircraft and is a 
safety flight issue. 

Northrop Grumman 

[El Segundo and Palmdale, California] 

Northrop Grumman is one 
of Lockheed Martin's princi- 
pal subcontractors. Northrop 
Grumman manufactures the 
center fuselage and weapons 
bay doors of the aircraft. 
The manufacturing opera- 
tions are divided between 
two manufacturing facilities: 
El Segundo and Palmdale, 
California. The engineering 
team is located at the 
El Segundo facility. That facility also produces the composite skins and weapons bay 
doors. The Palmdale facility develops and integrates the center fuselage component; 
including the tail cap antenna, composite panels and covers, in-flight operable doors. 
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arresting gear system, weapons bay door drive system, fire protection system, and 
multiple mission systems provided by its suppliers. Figure 5 shows the center fuselage 
on the integrated assembly line at the Palmdale facility. 

Our assessment of Northrop Grumman resulted in 66 findings that identified deficiencies 
in Northrop Grumman's implementation of the AS9100 Quality Management System. 
As part of our assessment of Northrop Grumman, we wrote several findings regarding 
the effectiveness of Lockheed Martin requirements flow down to suppliers. Figure 6 
provides the Northrop Grumman findings by AS9100 clause with the foUowing sections 
summarizing significant issues documented during the assessment. 

Figure 6. Northrop Grumman Findings 
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Planning of Product Realization (7.1) 

Our assessment documented several deficiencies in the area of Product Planning and 
Realization. We noted that the temperature and relative humidity parameters in the 
automated paint system were not programmed to required specifications, and operators 
changed the parameter limits without management approval. In two other instances, the 
required torque value was not included in the process work instructions. Specifically 
the torque paint application and special hole finish verification were not included in the 
process work instructions. A third finding documented a similar deficiency with adhesive 
application and cure times. We could not determine if these required parameters had 
been met or verified. Product reliability and repeatability relies on explicit process work 
instructions, and Northrop Grumman needs to assess the risk to and impact on delivered 
hardware resulting from these deficiencies. 

Customer-Related Processes (7.2) 

Customer Communication (7.2.3). Northrop Grumman did not provide DCMA with 
adequate access to Northrop Grumman command media or other enterprise systems 
to effectively administer its delegated contract oversight functions. Specifically, DCMA 
Palmdale requires access to corrective and preventative action databases, internal audits, 
metrics, policies, processes, and procedures. 

The lack of access to program Information prevented DCMA from effectively 
administering Northrop Grumman Palmdale contracts as required by the Federal 
Acquisition Regulation/Defense Federal Acquisition Regulation Supplement and 
monitoring JPO's expectations for cost, schedule, and performance. 

Design and Development (7.3) 

Design and Development Planning (7.3.1). We documented deficiencies with the 
software development processes for engineering software releases. Northrop Grumman 
released five engineering software versions of the mission systems software to Lockheed 
Martin without a formal documented product test, including the test plan for changed 
requirements and the test results. Instead, Northrop Grumman's practice was to test 
software releases only at the unit/module level. Northrop Grumman stated that software 
test performed at the unit level versus assembly level are done informally; therefore, 
no records were retained. In addition, the Software Safety Engineer was not actively 
participating in software product evaluations to review and assess safety assurance-level 
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requirements as required by the Software Development Plan. The assessment reviewed 
four software product evaluations; however, records indicated that the Software Safety 
Engineer had participated in only one of four evaluations. 

Software testing at the unit/module level creates an integration risk because modular 
testing may not have been sufficiënt to verify software interoperability issues. 

Purchasing (7.4) 

We cited deficiencies in Northrop Grumman's flow down of supplier requirements and 
supplier submittal of contract deliverable data. Some purchase orders showed incorrect 
part or material revision information. In one example, the purchase order for carbon fiber 
material did not include specification revision information. Northrop Grumman relied on 
the composite material supplier to determine the correct version of the ordered products. 
Receiving inspection personnel could not verify the correct versions of parts or materials 
received due to the lack of part revision information on the purchase order Therefore, the 
material was accepted and used in production without engineering approval or verification 
of correct version. Without revision information in purchase orders, receiving inspection 
personnel cannot verify that correct versions of parts or materials are received, and that 
incorrect parts and materials are not introduced into production. 

We sampled several of Northrop Grumman's automated purchase orders and found that 
the purchase orders did not include the means to identify specification revisions on raw 
material receiving reports. Because of the significance of this finding, Northrop Grumman 
initiated an in-depth investigation to determine the impact to delivered hardware. Other 
findings documented that Northrop Grumman was not foUowing its supplier management 
procedures. Suppliers rated with a "red," or high-risk supplier scorecard rating were still 
allowed to conduct self-inspections of material shipped to Northrop Grumman. Northrop 
Grumman should have imposed additional inspections to ensure material conformity. 

Production and Service Provision (7.5) 

Control of Production and Service Provision (7.5.1). Our assessment found evidence 
of incomplete manufacturing and quality assurance records and a lack of adherence to 
released process procedures. A major finding noted that Northrop Grumman did not 
ensure that all quality assurance verification operations were completed. Northrop 
Grumman did not complete several quality inspection/verification steps in shop orders 
for manufacturing the Integrated Power Package Door Mechanism and the Collector Panel. 
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We noted that Northrop Grumman personnel did not foUow released work instructions. 
The operators were not following the process work instruction for setting the composite 
cutting blade depth prior to the cut. The operator set a depth that did not agree with the 
work instruction. In another example, during center fuselage buildup operations, fuel 
sealant had been appUed to incorrect areas. The operator stated that the area sealed 
was no longer a "stay out zone" on the hardware; however, the work instructions had 
not been updated to reflect engineering's agreement to the change. In addition, two 
records documenting that the sealant appUcation on several CTOL tail fins had not been 
completed. Our assessment identified several more manufacturing deficiencies. 

• Composite cumulative adhesive shelf-life times were not recorded for some 
composite materials. 

• The liquid shim application process was not in compliance with 
documented procedures. 

• Drill bit life was not being tracked. Drill bit life should be tracked to ensure 
dimensional compüance. 

Production Process Verification (7.5.1.1). We noted that Northrop Grumman had not 
performed or completed FAIs on all F-35 center fuselage variant assemblies. A review 
of Northrop Grumman's FAI status and schedule data noted that FAIs were initiated 
but never completed for LRIP 4 variants. Northrop Grumman discontinued performing 
FAIs in accordance with Lockheed Martin's direction as stipulated in the July 20, 2011, 
released FAI plan. The Lockheed Martin FAIs Plan is in direct conflict with the SOW. 
Completion of FAIs on the F-35 center fuselage variant assemblies is essential to verify 
that the production process is capable of producing assemblies that meet requirements. 

Control of Production Equipment, Tools, and Software Programs (7.5.1.3). Our 

assessment noted deficiencies with the validation and control of numerical software 
programs. More than 75 percent of the F-35 composite numerical control machining 
programs were in an "unproven" state with no traceability established between the 
numerical control program and the part produced. Specifically the automated machinery 
screen used by the operator to select the file for machining listed numerous files, some 
listed as "proven," others as "unproven." Numerous files designated as "unproven" could 
be inadvertently selected to generate the exact same part numben A similar condition 
existed for the measurement programs used to validate the parts produced on the 
precision milling machine. These findings represent a significant risk that either 
unproven files or files not designated as the latest approved configuration could be 
selected and used in producing the product. 
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We documented tooi control deficiencies. The radio frequency identification system was 
incapable of tracking tooi location. It incorrectly identified tools as missing or in another 
location; allowing similar tools to be returned to incorrect locations. In addition, Northrop 
Grumman personnel did not always tracé missing tools as required by procedures. 
Several other findings noted deficiencies with the identification, storage, and application 
of tooiing. Ineffective tooi control can lead to quality issues such as the wrong tools used 
during assembly lost tools, and insufficiënt visibility in calibration measurement. It can 
also lead to field returns caused by uncalibrated equipment. 

Preservation of Product (7.5.5). Our assessment identified deficiencies with the control 
of limited-life materials. Northrop Grumman's process for controUing shelf-life materials 
did not ensure that all materials used on the manufacturing floor were within their useful 
life. Expired sealant material was used on flight hardware, and elastomeric filler was 
labeled with incorrect shelf life expiration dates. Also, 17 tubes of expired product were 
provided to the production floor We immediately notified Northrop Grumman, which 
initiated action to recall the nonconforming materials and began the material review 
board process for the affected parts. An additional finding noted that material shelf 
Hfe Information was not always identified on transfer containers, leading to a loss of 
traceability to its useful life. Using expired materials could affect structural integrity of 
the final product. 

FOD procedures were not being foUowed in production floor areas designated as 
category 1 FOD-critical areas. Specifically, a category 1 FOD-critical area requires 
heightened FOD awareness including cordoning off the area, displaying FOD-critical 
signs, and implementing sign-in/sign-out logs and tooi control logs. In one instance, 
crane operators in a FOD-critical area had not recorded themselves or their tooiing into 
the area. All tooHng brought into a FOD-critical area are required to be documented on a 
FOD-control card. In another instance, personnel were performing a close-out operation 
on an inverted fuel tank and side skins in a category 2 FOD-control area, but this operation 
should have been performed in a category 1 FOD-critical area. The purpose of a FOD 
program is to ensure thatloose hardware is accounted for, such as bolts or nuts, preventing 
the puncturing of tanks, hydraulic lines, or damaging other sensitive components during 
aircraft operation. 
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Control of Monitoring and Measuring Equipment (7.6) 

We noted Northrop Grumman was providing extension of calibration intervals without 
proper evaluation and engineering approval. A date extension was provided on a 
digital pressure gauge with a poor calibration history even though two out-of-tolerance 
conditions were documented on this item. Calibrated equipment with a history of out- 
of-tolerance conditions should have their calibration interval shortened, not extended. 

Other findings addressed calibration deficiencies such as the use of incorrect calibration 
procedures and ambiguous humidity requirements in the calibration lab. Without a 
disciplined and controUed calibration system, the accuracy and integrity of calibrations 
performed is uncertain. This may result in hardware being processed and accepted 
because the discrepant test and measurement equipment masked an out-of- 
tolerance condition. 

Monitoring and Measurement (8.2) 

Monitoring and Measurement of Product (8.2.4). We found that Northrop Grumman 
did not use the correct test system configuration or software configuration for performing 
electrical test methods on composite parts and material. Another finding documented 
that quality assurance personnel did not record or witness production measurements 
and test results on the integrated assembly line. We also noted that work instructions 
did not require quality assurance personnel to record nut plate push test results and rivet 
flushness measurements; therefore, no test values were recorded and test verification 
could not be done. Without proper test set-ups and quality assurance personnel 
verifying critical measurements, Northrop Grumman cannot be certain whether all the 
requirements were tested and verified. 

Conti nuous Improvement (8.5) 

Corrective Action (8.5.2). Our assessment team noted that the Northrop Grumman 
corrective action system for in-house operations and suppliers was ineffective. Northrop 
Grumman was in the process of transitioning into a new software problem reporting tooi; 
however, neither the legacy tooi nor the new tooi adequately captured the root cause 
and analysis of software nonconformance. We also noted that Northrop Grumman did 
not conduct root cause analysis for software internal audit findings and did not approve 
corrective action plans in a timely mannen Root cause analysis and timely corrective 
actions prevent future nonconformances and marginal products from being fielded. 
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Lockheed Martin Program Management at 
Northrop Grumman 

Our assessment documented several major findings related to Lockheed Martin's flow 
down of requirements to Northrop Grumman in the areas of Software Traceabiüty, 
Software Development, Software Corrective Action, and FAI. 

Lockheed Martin did not maintain mission systems requirements traceabihty to 
the software-level requirements. The Lockheed Martin Dynamic Object Oriented 
Requirements System [DOORS] requirements database contained about 300 orphan 
requirements for missions systems. An orphan requirement is one that is not derived 
from or traceable to top-level system requirements. Lockheed Martin identified this 
deficiency in July 2010; however, it did not document or implement the root cause and 
corrective action. 

Our review of the JSF Program Air System Block Plan showed that the plan had not been 
updated since August 13, 2008, and does not reflect current block planning defining 
Air Systems capabilities for the F-35 Program. As a result, there is no authoritative 
document defining current and planned F-35 software capabilities. 

Northrop Grumman Assessment Summary 

Our assessment found deficiencies at both the El Segundo and Palmdale sites. Deficiencies 
at the Northrop Grumman, El Segundo, site were primarily in the area of design and 
purchasing. Northrop Grumman's practice of performing unit/modular software testing 
without documenting the results creates significant concerns regarding requirement 
verification and interoperability. Without adequate design evaluation and test of mission 
system software, Northrop Grumman cannot ensure all software system and safety 
requirements have been met. Purchase orders lacked material version information to 
ensure that material was procured to the required specifications. After bringing this 
deficiency to their attention, Northrop Grumman initiated an investigation to determine 
the impact on the hardware. 
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Our assessment of the Palmdale facility noted several deficiencies, most of which were 
attributed to the creation of a new integrated assembly hne. Documentation of processes 
and procedures used in manufacturing had not reached the expected maturity. Procedures 
did not reflect the actual operation, were missing required detail, and required additional 
proofing for accuracy. Northrop Grumman had implemented a radio frequency identifier 
system for tooi control; however, it was ineffective and did not provide the necessary 
accountability and control warranted for a complex production and assembly area. 



BAE Systems 

(Samlesbury United Kingdom) 




Figure 7. BAE Aft Fuselage 

Source: Photo courtesy of BAE Systems 



BAE Systems is Lockheed Martin's 
other principal subcontractor 
BAE manufactures the aft fuselage 
and the carrier variant wing-fold 
system; including critical items 
such as the horizontal tails and 
vertical tails. Additionally BAE, 
as one of the major software 
suppliers for the F-35 aircraft, 

is responsible for developing fuel system software, navigation and display software, 
vehlcle system prognostics and health manager software, and off-board mission support 
software. Figure 7 is an image of the aft fuselage on the manufacturing floor at BAE. 

Our assessment of BAE resulted in 82 findings that identified weaknesses in BAE's 
implementation of the AS9100 Quality Management System. Additionally, Lockheed 
Martin had several deficiencies in requirements flow down to BAE. We also identified 
issues related to JPO and DCMA oversight. Figure 8 provides BAE findings by AS9100 
clause with the foUowing sections summarizing significant issues documented during 
the assessment. 
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Figure 8. BAE Findings 
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Documentation Requirements (4.2) 

Control of Documents (4.2.3). We identified several production documents that were 
not under configuration control. Examples of nonconfiguration controUed procedures 
and/or documents available for use include: chemical treatment, paint Identification, 
machining health and safety plan, control of heat treatment, and bore-cutting tooi 
and lubricate. 



BAE updated the F-35 quality assurance plan for LRIP 4 and 5, but did not submit the 
documents to Lockheed Martin for approval. Production process documentation not 
approved or under revision control included the folio wing checklists: daily production 
verification, first article verification, and production process verification. ControUing 
such documentation creates the means to ensure each assembly is exact and conforming 
to requirements. 
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Human Resources (6.2) 

Competence, Training, and Awareness (6.2.2). BAE lacked discipline in the areas of 
employee training requirements, recertification, and training records. Skills matrices 
did not always identify the training required for each operator to perform a particular 
manufacturing process. For example, within building 355, no one had completed training 
competency in any of the required technical skills for the foUowing critical processes: 
press operations, friction sawing, part marking, tungsten inert gas welding, automatic 
grit blast machining, and radial arm drilling. Many of the personnel lacking training 
were responsible for manufacturing fracture-critical hardware. In addition, we found in 
the other manufacturing facilities that BAE had not established performance criteria to 
evaluate employee competency. We also found that BAE did not have a recertification 
program for wire harness repair. 

A training and certification program, to include control of training records, is critical to 
ensure that personnel manufacturing and inspecting hardware are proficient to prevent 
nonconformances. BAE's lack of a disciplined training program is a significant concern to 
the production of F-35 hardware. 

Planning of Product Realization (7.1) 

Configuration Management (7.1.3). BAE suppliers were not submitting minor 
changes to BAE for review and approval as required. For example, Honeywell, a BAE 
supplier, approved an engineering change request for the F-35 Life Support System 
without Lockheed Martin or BAE approval. Another BAE supplier, Ametek, approved an 
engineering change order for coating painting processes without BAE approval. 

The assessment also noted that BAE suppliers were not submitting configuration 
status accounting records or reports that provide the "as-built" versus "as-designed" 
configuration. For example, Honeywell did not provide a configuration status accounting 
report for products delivered during LRIP 4 and LRIP 5. Additionally, Goodrich 
Actuation also had not provided a configuration status accounting report for LRIP 5 
delivered products. 

BAE did not have schematics/drawings to define the Fuel Management System 
software test stand and did not have the set-up/take-down procedures or 
troubleshooting guidelines necessary to maintain the test stand configuration. BAE 
also did not conduct software certification of the software test stand, as required 
prior to software qualification testing. 
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Controlling engineering changes and providing adequate configuration status accounting 
of product is essential to ensure the approved configuration replicated correctly every 
time. Failure to maintain the required configuration management disciphnes increases 
program performance, cost, and schedule risks. 

Design and Development (7.3) 

We identified several major deficiencies in the areas of program management, 
configuration control, and testing. In the area of software development, BAE was 
working to an outdated version of the F-35 mission systems software development 
plan [SDP),^ was not maintaining the software design document [SDD] for the vehicle 
system fuel management system, and was working to unapproved software regression 
test procedures. In the area of hardware development, BAE was not approving all design 
review plans for the F-35 Program. 

Design and Development Planning (7.3.1). BAE developed F-35 software 
without using the latest Lockheed Martin approved SDP. According to the contract, 
BAE is required to follow the latest software design plan released by Lockheed Martin. 
BAE was developing software to the Mission Systems Software Development Plan 
revision C [dated September 15, 2007), although the current approved and released 
revision is revision E [dated August 14, 2012]. Inadequate flow down of software 
development requirements could affect aircraft performance and safety requirements. 

Design and Development Outputs (7.3.3). BAE's software requirements specification 
[SRS] for the vehicle system fuel management system was revised 15 times over a 5-year 
software development period; however, the SDD was not updated with these changes. 
For example, the SDD was not updated to reflect corrective actions for fuel management 
system software trouble reports that were in the revised SRS. Software design decisions 
and requirements changes should be captured in the SDD; otherwise, the software design 
baseline may not meet system performance or safety requirements. 

Design and Development Review (7.3.4). We noted that the life cycle management 
design review plan had not been approved by the project engineering manager and 
the design review chairman. Approval of the plan certifies that the objectives and exit 
criteria fulfill the objectives of the review. If the objectives and exit criteria in the design 
review plans are not certified, there is no assurance that performance requirements will 
be realized. 



^ The SDP describes a developer's plans for conducting a software development effort. The term "software development" is 
meant to Include new development, modification, reuse, reengineering, maintenance, and all other activities resulting in 
software products. (DAU) 
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Design and Development Validation (7.3.6). We documented that BAE was using 
an unapproved regression test procedure and conducted incomplete regression tests. 
Regression testing is important because it provides confidence that changes do not 
degrade functionality. One finding documented that there was no approved regression 
test procedure for the Prognostics and Health Management software release. Additionally, 
the functional test description for the Prognostics and Health Management software 
requires 35 test runs, hut the test report showed that BAE partially conducted only 9 of 
the 35 required tests. In addition, in the software requirements verification matrix, 
four requirements were determined "unverifiable by test" and required verification by 
analysis; however, the analysis was not performed. 

Purchasing (7.4) 

BAE's flow down of requirements to suppliers was insufficiënt to ensure that the 
delivered product meets all the necessary quality assurance and technical requirements. 
The assessment documented several findings citing inadequacies in BAE flow down of 
supplier requirements and suppliers submittal of contract deliverable data. 

Purchasing Information (7.4.2). BAE issued several purchase orders to suppliers that 
contained inaccurate and inconsistent requirements. For example, two purchase orders 
cited the wrong SOW, and three purchase orders did not specify Government-furnished 
equipment by part number. 

BAE also flowed down incorrect configuration management requirements to suppliers. 
In accordance with Lockheed Martin direction, the "Configuration Management 
Requirements for F-35 Suppliers and Subcontractors," should have been flowed to 
suppliers in purchase orders supporting LRIP and foUow-on contracts. However, an 
earlier requirements document, titled, "F-35 Joint Strike Fighter Program, Configuration 
Management Plan," had been flowed to suppliers, but did not contain the necessary 
supplier configuration management responsibilities and requirements. Without proper 
flow down of technical and quality assurance requirements, BAE cannot ensure that their 
products meet performance, reliability, and other system requirements. 

Verification of Purchased Product (7.4.3). BAE was not verifying that its suppliers 
were submitting critical supplier data deliverables such as acceptance test procedures 
[ATPs), quality assurance plans, and configuration management certifications. 
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Production and Service Provision (7.5) 

Control of Production and Service Provision (7.5.1). We documented that BAE was 
not following process inspection procedures, completing manufacturing and quality 
assurance records, or complying with tooi control processes. BAE Standard procedures 
require that inspectors verify and record that operations are complete. However, 
in several instances, the inspector did not record completion of the operations. For 
example, we documented that quality assurance personnel had not verified epoxy primer, 
urethane topcoat, and abrasion-resistant coating processes. In another case, there was 
no indication that inspectors had completed buy-off of the water break test (ensures that 
the surface is properly prepared prior to bonding] on the tail-hook door and aft section. 

Several procedures did not contain adequate detail or requirements to control the 
process. For example, specification requirements for the application and mixing times 
for the epoxy primer and topcoat were incorrectly identified on the paint shop process 
documents. As a result, actual mix times recorded in manufacturing process documents 
did not meet the specified requirements. 

We found evidence of incomplete manufacturing and quality assurance records. 
In 15 instances, cleaning and cure details were not recorded in the mix log books. We found 
records with insufficiënt details for paint pot life, mix time, and touch-up processes. Mix 
times and pot life details are essential to ensure that material properties meet specification 
requirements prior to application. The failure to document manufacturing and inspection 
operations can lead to incomplete work, loss of traceability and nonconforming products. 

Production Process Verification (7.5.1.1). A major finding noted that BAE had not 
performed or completed FAIs on all F-35 details, subassemblies, and assemblies as 
required by Lockheed Martin supplier requirements and BAE internal instructions. For 
example, 11 subassembly FAIs were not conducted and three subassemblies had failed 
the initial FAI. A corresponding major finding was directed to Lockheed Martin for 
flowing down conflicting FAI requirements. Successful completion of FAIs on F-35 details, 
subassemblies and assemblies is one of the criteria for LRIP; therefore, BAE's processes 
do not reflect that criterion. 

Control of Production Equipment, Tools, and Software Programs (7.5.1.3). The 

assessment identified several major deficiencies documenting the inadequate control 
of manufacturing tooiing, jigs, and fixtures. Operators applied an unapproved steel 
epoxy stick in the incorrect location to hold two parts on the aft closeout rib in position 
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for machining operations rather than using the specified tooiing. We found that BAE 
personnel were not monitoring tooi wear of cutting tools in the aft fuselage assembly 
in accordance with their production work instructions. To prevent quality defects, the 
maximum specified use for a drill bit was 50 drilled holes; however, operators were not 
tracking the number of times a drill bit was used. 

We documented systemic issues with the storage, inventory, and accounting of tooiing. 
Tools were often unaccounted for and not in their correct locations as designated by the 
work cell tooi inventory sheets. A major finding documented that BAE personnel did 
not perform visual, functional, and dimensional tooi cycle checks on the final assembly 
tooi fixture as required. Failure to control manufacturing tooiing can lead to the use of 
discrepant, damaged, or incorrect tooiing. 

Identification and Traceability (7.5.3). We documented that inventory lists located on 
carts throughout the factory were inaccurate. We also found hardware commingled, and 
in some cases, improperly identified. As a result, positive part identification and control 
through the assembly process was not always evident. Because parts are manufactured 
for a specific airframe and are not readily interchangeable, failure to maintain traceability 
and identification of those parts results in interchangeability issues, nonconformances, 
and unnecessary rework or scrap. 

Preservation of Product (7.5.5). The assessment documented numerous examples 
of limited shelf-life materials [LSLM) improperly labeled, tracked, and controUed. For 
example, BAE personnel used expired Click Bond adhesive material in the manufacture of 
vertical and horizontal tail assemblies. Other adhesive materials labeled with improper 
expiration dates were available for use. In addition, a bottle of expired alodine material 
was used on the production floor in the carbon subassembly area. Shelf-life material that 
is not properly labeled or controUed can lead to the use of expired materials in production. 
Use of expired shelf-life material compromises the assurance that its material properties 
will satisfy the intended requirements and the integrity of the assembly. 

Control of Monitoring and IVIeasuring Equipment (7.6) 

BAE was not controUing calibration laboratory processes to meet established 
requirements. BAE was operating two calibration laboratories, but neither laboratory 
was accredited to meet ISO/IEC (International Electrotechnical Commission] 17025, 
"General Requirements for the Competence of Testing and Calibration Laboratories," as 
stipulated by BAE's internal procedure and a Lockheed Martin requirement. In addition. 
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several calibration procedures in the Test Instrumentations Laboratory did not address all 
requirements and criteria required by American National Standards Institute (ANSI] Z540, 
"Requirements for the CaHbration and Measuring and Test Equipment." For example, 
the caübration procedures for temperature sensors and field test instrumentation did 
not address measurement ranges and tolerances for the item to be calibrated, minimum 
performance requirements, and criteria and requirements for calibration. Moreover, BAE 
did not have procedures to calculate the uncertainty of measurements for calibrations 
performed or a calibration procedure for furnaces or ovens. 

BAE was not following its internal procedures for the tagging and removal of expired 
monitoring and measurement equipment. Numerous examples of equipment with 
calibration deficiencies were found throughout the facility. Deficiencies included out-of- 
calibration torque wrenches, vernier calipers, and a surface plate. Furthermore, torque 
wrenches were used for verification outside of their calibrated capability. In addition, BAE 
was not assessing the effect of out-of-tolerance equipment used on production hardware. 
In summary the number of tooi control and calibration deficiencies leaves uncertainty in 
BAE's product processing and acceptance. 

Measurement, Analysis, and Improvement (8.0) 

Control of Nonconforming Product (8.3) and Corrective Action (8.5.2). BAE is 

closing material review board documents without confirming effective and meaningful 
corrective action. Specifically four out of ten material review board dispositions for 
nonconformances were closed without implementing the specified corrective action. In 
another finding, BAE did not submit all material review dispositions to the onsite DCMA 
representative for concurrence. Specifically, a waiver/deviation for the vertical tail 
substructure included both nonstandard repairs and use as is dispositions without any 
evidence of submittal to the DCMA representative for required approval. 

The supplier corrective actions process did not always ensure suppliers responded to 
corrective action requests within the required time period. For example, 21 of 63 supplier 
corrective action request forms were past due by as much as 200 days. Furthermore, the 
supplier corrective action data were not accurately maintained. The supplier electronic 
corrective action database identified several supplier corrective action requests as closed; 
however, a sample review of the documentation indicated the status as open. Effective 
review and verification of material review board dispositions and associated corrective 
actions are necessary to correct and prevent nonconformances. 
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Lockheed Martin Program Management at BAE 

We reviewed the effectiveness of Lockheed Martin program management and wrote 
seven findings. Lockheed Martin did not communicate to BAE that the latest software 
development plan was revision E, released on August 14, 2012; thus, BAE was developing 
JSF Mission Systems Software to revision C, dated 2007. The difference between the C and 
E revision could have a significant impact on functionality and performance. 

In another finding, the F-35 Air Systems Software Development Plan requires that each 
product-level software development plan include plans for software maintenance. 
However, the Lockheed Martin approved software development plans for Utilities and 
subsystems, prognostics and health management, ejection seat sequencer, boost/ 
transfer pump controller, and life support system did not include the required software 
maintenance section. As a result, there is no assurance that software released is 
maintainable after delivery. 

Lockheed Martin's FAI plan conflicts with the AS9102 requirements flowed down to BAE 
in the SOW, resulting in BAE not performing FAIs on major assembhes and subassemblies. 
Failing to perform FAIs in accordance with AS9102 could result in the inability to achieve 
process stability and successfully meet program production rate goals. The lack of 
process stability could also result in increased costs resulting from discrepant hardware 
and/or the need to perform additional product conformance verification to ensure 
engineering requirements. 

Lockheed Martin did not flow down the correct quality management system requirement 
and the correct nonconforming material retention requirements. Additionally, Lockheed 
Martin did not specify the delivery requirements for the Counterfeit Parts Prevention 
and Control Plan; therefore, BAE did not produce a plan as required by contract. The 
Lockheed Martin configuration management plan imposed on BAE identifies additional 
requirements for BAE to foUow; however, several of those references and the processes 
were no longer valid. Flow down of current and correct requirements to suppliers is 
essential to ensure delivered products meet program requirements. 

Joint Program Office Oversiglit at BAE 

JPO did not flow requirements for a CSI program to Lockheed Martin; therefore, BAE 
did not implement the required processes to ensure safety of flight. 
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DCMA Oversight at BAE 

The DCMA UK was not reviewing BAE classifications and dispositions of all minor 
variances, nor participating in or reviewing the BAE Material Review Board actions as 
required by its LOD. BAE processes an average of 20 minor variations a day; however, 
as agreed to with BAE, DCMA reviews only about 10 percent of these, or 40 variances a 
month. In addition, DCMA UK had not delegated the responsibiUty for review of minor 
variances to DCMA office located at BAE suppUers. Government oversight ensures that 
engineering evaluations and corrective actions occur. 

BAE Systems Assessment Summary 

The software tests review provided üttle confidence that software was fully tested. There 
was also inadequate communication and requirements flow down from Lockheed Martin 
to BAE, leading to software configuration management gaps. 

SuppHer management practices failed to consistently define technical and quaUty 
assurance requirements in purchase orders, ensure suppUer corrective actions were 
completed within process timeframes, and verify suppHer data submittals were in 
accordance with purchase orders. BAE must ensure all technical and quality assurance 
requirements are properly included in contracts and implemented by their suppliers to 
meet cost, schedule, and performance goals. 

In addition, improved production discipline is required. The assessment showed that 
additional process discipline is required to ensure conformity to end-item requirements. 
Personnel were relying on their knowledge of the design and production processes, 
rather than following documented processes. We found 14 examples of documentation 
that lacked essential detail to define, implement, and capture production processes. 

Tool control, maintenance of precision tools, and calibration needs attention. Uncontrolled 
and unaccounted for tools were prevalent throughout the production facility. Additionally, 
the calibration system had significant deficiencies. A disciplined calibration program is 
essential in determining the performance and acceptance of product. 



36 I DODIG-2013-140 



Contractor Assessments 



L-3 Display Systems 

[Alpharetta, Georgia) 

L-3 Display Systems (L-3], Alpharetta, Georgia, a division of L-3 Communications 
Corporation, is a Lockheed Martin subcontractor that manufactures the Panoramic 
Cockpit Display (PCD] for the F-35 Lightning aircraft. The PCD, Figure 9, performs 
significant functions such as providing pilot flight navigation and sensor information, 
communication, and threat situational awareness. The PCD consists of two major 
components, the Display Unit (DU] and the Electronics Unit (EU]. The DU is the interface 
between the pilot and the aircraft situational awareness systems while the EU is the 
electronics package capturing and providing data to the DU from aircraft systems. 

Figure 9. L-3 Communications Panoramic Cockpit Display. 
Source: Image courtesy of L-3 Display Systems 




Our assessment of L-3 resulted in 56 findings that identified deficiencies in L-3's 
implementation of the AS9100 Quality Management System. Figure 10 provides L-3 
findings by AS9100 clause with the foUowing sections summarizing significant issues 
documented during the assessment. 



DODIG-2013-140 | 37 



Contractor Assessments 




Documentation Requirements (4.2) 

Our assessment of L-3's document control system identified several findings with review, 
revision control, approval, and distribution of items such as internal procedures and 
manufacturing instructions. We found some documentation did not have management 
approval, were released without periodic review, had inaccurate revisions and dates, or 
document indexes that were not maintained. Check sheets used to define the buildup 
of circuit cards in the surface-mount technology assembly area were not controUed 
and there was no means to verify the latest revision. This lack of accurate, complete, 
and current documentation could adversely affect the ability of the F-35 Program to 
deliver products that meet customer performance requirements in accordance with 
Schedule requirements. 
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Configuration Management (7.1.3) 

Our assessment documented several findings pertaining to configuration management. 
We identified a lack of baseline technical documentation for L-3 DU and EU products. 
According to the L-3 product configuration documentation, a documented product 
baseline was required concurrently with the first production aircraft delivered in 
May 2011; however, this was not done. Similarly, an allocated baseline configuration was 
required for CDR closure, but it also was not done. A configuration baseline establishes a 
product's form, fit, and functional performance that is analyzed, tested, and characterized. 
Such a rigorous approach was lacking, and could result in additional product cost. 

L-3's engineering change control process lacked discipUne. L-3 incorrectly classified 
engineering changes that affect the form, fit, and function as "minor" changes when they 
should have been classified as "major." Major engineering changes require Lockheed 
Martin approval. For example, L-3 classified two changes as "minor" that increased the 
weight of the display interface circuit card assembly and weight of the touch screen by 
85 percent and 62 percent, respectively. These changes affected product form, fit, or 
function and should be classified as "major" and not as "minor." We identified that L-3 
implemented several "major" changes prior to receiving Lockheed Martin authorization. 
Incorrect classification of engineering changes and the implementation of engineering 
changes prior to authorization bypass the systems engineering review and approval 
cycle. Inadequate review of engineering changes will adversely affect product performance, 
quality, and reliability. 

Design and Development (7.3) 

We found that the electromagnetic interference [EMI] qualification for the PCD, EU, and 
DU did not meet the performance-based specification, frequency bandwidth requirement. 
The performance-based specification requires that radiated susceptibility testing 
be performed over a specific range of frequencies [10 KHz to 18 GHz) in accordance 
with MIL-STD-461E, "Requirements for the Control of Electromagnetic Interference 
Characteristics of Subsystems and Equipment." However, L-3 was performing testing in 
the range of 30 MHz to 18 GHz without engineering approval for the reduced frequency 
range. Although the EMI test report was comprehensive, the quality assurance review 
process failed to identify that a performance-based requirement was not being met. 
Additionally, according to the documentation, the EMI qualification test was completed 
without issues. 



DODIG-2013-140 | 39 



Contractor Assessments 



L-3 did not adequately review the F-35 PCD integrated system verification plan/ 
procedures. The PCD Functional Test Procedure, Software Functional Test Description, 
and Requirements Verification Matrix had not been reviewed or dispositioned. We 
also found that Lockheed Martin, as the prime contractor, should have approved these 
documents hut did not. 

Product and Service Provisions (7.5) 

Control of Production and Service (7.5.1). Quaüty assurance personnel did not verify 
several DU ATP luminance^ requirements to the updated requirement. Specifically black 
nonuniformity and red, green, blue, and white luminance nonuniformity were required 
not to exceed the value of 30 percent. L-3 indicated to Lockheed Martin that it could 
not reasonably meet the specification, so Lockheed Martin issued a variance allowing the 
value not to exceed 50 percent. Although Lockheed Martin granted requirement reliëf, 
the L-3 test procedure did not reflect the change, and quality assurance personnel did not 
verify the new requirement. 

Production Process Verification (7.5.1.1). Our findings identified deficiencies with 
L-3's supplier FAI program. L-3 did not require all selected suppüers to perform FAIs. 
Specifically, the JSF contract provision for the LRIP 5 states that FAI requirements shall 
be flowed down to L-3 suppliers. Two purchase orders for suppliers, Ayesas and DDI 
Sales Corporation, required FAI on their product. However, the supplier did not perform 
FAIs, and L-3 accepted the hardware without suppliers demonstrating the capability to 
produce conforming products. As an example, we found a supplier FAI report containing 
several nonconformances that were not addressed, although L-3 had been accepting 
hardware from this supplier since 2008. Finally L-3 was not maintaining all supplier 
FAI quality assurance records in accordance with contractual requirements. L-3's lack 
of systems engineering, regarding FAIs, creates uncertainty that components built in the 
future will conform to requirements. 

Control of Production Equipment, Tools, and Software Programs (7.5.1.3). 

L-3 did not maintain EU and DU test stations to drawing requirements. Of the 
five test stations in operation, one test station did not have a released drawing package 
defining the equipment, equipment wiring, software, and verification of accuracy. Another 
test station had a released drawing but the test station was not configured in accordance 
to the released drawing. In addition, we could not find evidence that similar test stations 



Merriam-Webster dictionary defines luminance as: "the luminous intensity of a surface in a given direction per unit of 
projected area." 
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would produce comparable results. We found the method for performing correlation 
analysis among the test stations was undefined. Correlation studies should be performed 
to quantify the variabiUty between multiple test stations used for final acceptance test. 

Preservation of Product (7.5.5). Our assessment team identified four findings 
regarding the control of LSLM, FOD, and electrostatic discharge (ESD) processes and 
practices. We found LSLM items on the manufacturing floor not stored at the required 
temperature, improperly identified, or expired. Additionally L-3 did not fuUy implement 
or foUow its product protection program and requirements for FOD and ESD. DUs were 
stored on transport carts without adequate FOD and ESD protection. In addition, L-3 
did not perform verification of the conductive flooring and constant wrist strap monitors 
in accordance with its ESD Compliance Verification Plan. This lack of preservation of 
product could result in reduced product quality and reliability. 

7.6 Monitoring and Measurement Equipment 

L-3 could not demonstrate adherence to calibration standards. The current Lockheed 
Martin purchase order requires L-3 to have a calibration program that meets one of the 
three industry Standard requirements: ISO 10012-1, "Requirements for Measurement 
Processes and Measuring Equipment," ISO 17025, "General Requirements for the 
Competence of Testing and Calibration Laboratories," or ANSI Z540-3, "Requirements for 
the Calibration of Measuring and Test Equipment." 

We found seven examples of out-of-calibration equipment still in use on the production 
floor or available for use. Four pieces of equipment were in the calibration area and 
not properly identified and segregated. Three others were available for use on the 
production floor with only one of the three tagged as out-of-calibration. L-3 could not 
provide objective evidence that an out-of-calibration piece of equipment was traceable 
to the flight units tested, as evidenced by evaluating three flight unit acceptance records. 
L-3's calibration and control system failed to provide adequate corrective actions. As an 
example, engineering personnel evaluated an out-of-tolerance photometer requiring the 
next four photometers to be evaluated to the internal lighting Standard. We found no 
documentation or evidence that the evaluation had occurred. 
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L-3's dimensional final inspection methods and measurement equipment were 
incapable of verifying product conformance to drawing requirements. The ATP required 
the inspector to use "single-dimension vernier cahpers" to measure a multi-dimension 
geometrie parameter on the DU. However, this measurement equipment and method was 
incapable of measuring the required dimensions. 

Our findings indicate that the systemic issues with the control of monitoring and 
measurement equipment could significantly affect measurement accuracy and/or quality 
assurance data used to determine product acceptance. 

Lockheed Martin Program Management at L-3 

We reviewed the effectiveness of Lockheed Martin program management because it is 
the prime contractor ultimately responsible to integrate L-3 products. We found that 
the performance-based specification for the aircraft PCD flowed down from Lockheed 
Martin to L-3 had incomplete, conflicting, and ambiguous system requirements. There 
were action items from CDR pertaining to the requirements that were not formally 
closed, leading to the document ambiguity. The Lockheed Martin corrective action 
system implemented at L-3 failed to effectively eliminate nonconformities and prevent 
their recurrence. Lockheed Martin accepted PCDs without L-3 having performed FAI 
and did not require L-3 to perform configuration status accounting for components 
delivered for LRIP units 1 through 5, that were previously delivered. Lockheed Martin 
is not approving or disapproving all major changes nor providing concurrence on the 
classification submitted by L-3, providing further evidence that Lockheed Martin was not 
sufficiently managing its suppliers. 

DCMA Oversight at L-3 Communications 

We found that DCMA Fort Worth, the prime DCMA office for the program, and DCMA 
Atlanta did not execute all responsibilities specified in the memorandum of agreement 
for the F-35 Program. DCMA Fort Worth delegated the administration of Government 
contract quality assurance of all F-35 parts produced at L-3 to DCMA Atlanta in a letter 
of delegation (LOD), with the requirement that Atlanta perform risk-based surveillance 
on high-risk components. During our review of DCMA Atlanta records, we found no 
evidence that surveillance was being conducted on F-35 products within the L-3 facility. 
Furthermore, DCMA Atlanta was not providing the prescribed reports requested in the 
LOD such as quality escapes, supplier risks, corrective action requests, and monthly 
Product Assessment Reports. 



42 I DODIG-2013-140 



Contractor Assessments 



L-3 Communications Assessment Summary 

We found deficiencies in most of the AS9 100 Quality Management System disciplines. The 
most significant deficiencies highUght the lack of process discipHne in implementation of 
Product Reahzation, which includes the following categories: configuration management, 
product verification, and monitoring and measurement equipment. 

The implementation of L-3's configuration management system was inadequate and 
could lead to inconsistencies in product performance, functionality, and reliability. The 
L-3 DU and EU development program did not produce product configuration baselines 
prior to delivery of the first production unit. L-3 had incorrectly classified engineering 
changes that affected form, fit, and function, such as increases to weight and human 
interface constraints, and had not received Lockheed Martin approval for major design 
changes prior to manufacturing the product. 

L-3's lack of process discipline in the area of product verification was particularly 
disconcerting. We noted systematic breakdowns in requirements verification, FAL 
test station configuration, and preservation of product that raises questions about the 
integrity of the L-3 product. L-3 did not consistently verify that product acceptance test 
data met specified requirements and did not consistently review FAI reports. In addition, 
L-3 did not manage or maintain test station configurations. L-3 also insufficiently 
implemented LSLM, FOD, and ESD processes. 

L-3 did not demonstrate adherence to calibration standards. L-3 did not ensure out- 
of-calibration equipment was identified and removed from the production floor L-3's 
calibration system failed to recall flight hardware after discrepant test equipment 
was identified. 
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Honeywell Aerospace 

[Yeovil, United Kingdom) 

Honeywell is a subcontractor to BAE and a tier 2 subcontractor to Lockheed Martin. 
Hone3well manufactures the F-35 aircraftlife support system, consisting of the On-Board 
Oxygen Generation System (OBOGS), Backup Oxygen System (BOS), and Service 
Connection Package. Figure 11 is a diagram of the Life Support System. 

Figure 11. Diagram of the Life Support System. 
Source: Image courtesy of Honeywell 




Our assessment of Honeywell resulted in 38 findings that identified deficiencies in 
Honeywell's implementation of the AS9100 Quality Management System. Figure 12 
provides Honeywell findings by AS9100 clause with the foUowing sections summarizing 
significant issues documented during the assessment. 
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Human Resources (6.2) 

Competence, Training, and Awareness (6.2.2). The Honeywell training program was 
inadequate. Honeywell does not have an overarching procedure to govern the training 
requirements, training roles and responsibilities, and records retention requirements. 
The Human Resource Manager explained that although Honeywell delegated the various 
assembly floor training needs and records to the respective floor managers; in practice, 
each manager had a different approach. As a result, not all records and certifications were 
retained or readily accessible. Additionally we noted personnel were manufacturing 
F-35 hardware with expired soldering and ESD certifications. Training ensures personnel 
proficiency in industry best practices, techniques, and methods. Without a rigorous 
training program, Hone3well cannot ensure the products delivered to BAE conform to 
expected workmanship standards. 



DODIG-2013-140 | 45 



Contractor Assessments 



Planning of Product Realization (7.1) 

Configuration Management (7.1.3). Honeywell did not control software configuration 
changes made to the oxygen monitor software. Hone3rwell was making changes to the 
released oxygen monitor software using the developmental change control processes 
without the necessary reviews, classifications, and verifications required for released 
software. We found that changes were approved without identifying the actual 
change implemented, the severity of change, the impact on software requirements or 
specifications, and the level of retesting required for the change. Honeywell's software 
configuration control process poses a risk to system integration and performance because 
the changes were not evaluated by engineering and management. 

Design and Development (7.3) 

Design and Development Inputs (7.3.2). Hone3well conducted Preliminary Design 
Reviews (PDRs] and CDRs for the life support system, chemical biological filtered air 
system subassembly and pilot interface connector without approved specifications. We 
noted that the Oxygen Monitor system software functions were not traceable to system 
requirements and have not been validated or verified. These functions included self- 
diagnostics, monitoring and control, and operating system interface standards. Without 
requirements definition, traceability and verification; Honeywell cannot ensure all the 
F-35 requirements for production have been met. 

Design and Development Outputs (7.3.3). Honeywell's software Version Description 
Document [VDD)^ for the Oxygen Monitor system software does not adequately reflect 
the software configuration changes implemented and lacks other pertinent information 
regarding the released software build. We found that the VDD did not identify software 
configuration changes in the software baseline, did not state the new firmware 
configuration tested, and did not reference the required interface specifications. 
As a primary document used for configuration Identification of the software, it 
lacked the principal elements to provide BAE sufficiënt insight into Honeywell's 
software configuration. 

Design and Development Review (7.3.4). We documented that the chemical 
and biological filtered air supply test readiness review was not conducted prior to 
qualification testing in accordance with Honeywell's procedure. The purpose of a test 
readiness review is to provide management with the assurance that the requirements. 



^ Institute of Electrical and Electronics Engineers STD 610.12 defines VDD as, "The document that accompanies and 
identifies a given version of a system or component. Typical contents include an inventory of system or component 
parts, identification of changes incorporated into this version, and installation and operating information unique to the 
version described." 
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proposed method of verification, the test methods, and equipment to be tested are 
mature enough to commence testing. We also found that there were still 13 of 14 
open action items from CDR. Open action items included updating the technical 
procurement specification to include acceptance testing and cleanhness level 
requirements, obtaining critical technical analysis for system safety and determining 
how to protect against contamination and electromagnetic interference. Honeywell 
management elected to proceed into initial production without the chief engineer's 
approval in accordance with their design review procedure. 

Purchasing (7.4) 

We found circuit board assembly purchase orders that did not include the 
supplemental requirements for electronic and electrical components, such as defining 
lead and lead-free finishes, electronic solder requirements, and electronic markings. We 
also found that personnel were not verifying product technical data sheets as part of the 
product acceptance process. Specifically, Hone3well accepted a pressure switch gage 
using only a certificate of conformance even though an acceptance test data sheet was 
also required for acceptance. The lack of supplier performance requirement verification 
could result in quality escapes. 

Production and Service Provision (7.5) 

Control of Production and Service Provision (7.5.1). We found that personnel 
responsible for performing sampling of supplier components were not following the 
required sampling plan. For instance, the sampling plan required a minimum sample 
of 20 items; but only 1 item was sampled. In another instance, an operator removed a 
protective film prior to machining operations in contradiction of the drawing notes. 

We found that the procedures for handling of environmentally sensitive material were 
insufficiënt. Honeywell personnel were not: recording and monitoring sealing compound 
pot life, recording high-strength adhesive film freezer out times, and recording and 
monitoring cure times for sealing compounds. Additionally Honeywell personnel did not 
maintain production equipment and tooiing in accordance with established procedures: 
soldering irons were not checked on a monthly basis, visual and leak test record sheets 
were not kept, equipment safety checks were not always performed, and production 
tooiing was inadequately identified. 

Preservation of Product (7.5.5). Honeywell was not ensuring personnel were 
following ESD and FOD procedures in the manufacturing and test areas. Oxygen monitor 
controllers were not stored in ESD-approved bags, static generators [e.g., a computer 
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keyboard, telephone, and computer monitor) were in close proximity to ESD mats, and a 
nonconductive production floor was used where ESD-sensitive items were processed. In 
the FOD-control area, we found that tools were not placed back on shadow boards and 
there was no accountability of piece parts [small nuts, bolts). Industry experience has 
shown that strict adherence to ESD and FOD procedures can prevent costly latent defects 
and hardware nonconformances. 

Control of Monitoring and Measuring Equipment (7.6) 

Honeywell's control of monitoring and measuring equipment was inadequate as 
evidenced by the foUowing: uncontrolled environmental conditions in the calibration 
laboratory and processing areas; uncalibrated tools in production, test, inspection, and 
metrology areas; and no maintenance of software for measuring equipment. We found 
uncalibrated pin gages in the receiving inspection area, a digital voltmeter in the test lab, 
air pressure gauges in the OBOGS assembly area, and a supply pressure regulator in the 
calibration lab. 

Control of Nonconforming Product (8.3) 

Honeywell was not accurately completing quality notifications (QNs] used to 
document hardware nonconformances. We found that two nonconformances were 
dispositioned as repaired; however, a repair had not occurred. Honeywell also 
dispositioned another QN without documenting the root cause and corrective action. 
Accurate QNs are necessary to prevent recurrence of the nonconformance. 

Analysis of Data (8.4) 

Honeywell personnel implemented acceptance test procedural changes without the 
required BAE review and approval for the oxygen concentrator and OBOGS. Additionally 
several ATP waivers/deviations were incorrectly categorized as minor rather than 
major. Requirement changes defined in either ATPs or waivers/deviations directly affect 
product performance and should be evaluated by BAE engineering. 

BAE Systems Program Management at Honeywell 

In performing quality assurance assessments of a supplier, we identified some findings 
against Honeywell's customer, BAE. We documented that BAE's SOW to Honeywell 
for LRIP 4 and LRIP 5 were not under configuration control and did not contain the 
appropriate document number. As a result, SOWs could not be traced to the appropriate 
contract, creating a risk that Honeywell may be manufacturing products to obsolete or 
incorrect program requirements. 
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Honeywell Assessment Summary 

Our assessment indicated an immature product design and production processes. 
The design review process allowed the OBOGS program to enter CDR without firm 
specifications, exit CDR with 13 open actions, and allowed production to proceed 
with unapproved design documents. Additionally, verification of hardware and 
software performance requirements was not conducted to the approved acceptance 
test requirements. 

Honeywell failed to communicate effectively to BAE the impact of changes and 
variances to the design. In several instances, Honeywell failed to obtain the necessary 
review and approval from BAE for changes to the Oxygen Concentrator acceptance test 
procedures and Oxygen Monitor software systems. Additionally, several variances were 
incorrectly categorized as minor when they should have been major, and should have 
been submitted to BAE for approval. 

Process discipline issues were prevalent throughout the facility. We found numerous 
procedural and implementation issues in the areas of ESD, FOD, calibration, and control of 
LSLM. The issues highlight quality assurance and process discipline concerns that could 
introducé unscheduled rework and repair, leading to cost overruns and schedule slips. 

United Technologies Corporation, Aerospace Systems 

[Fort Worth, Texas, and Independence, Ohio) 

UTAS is a Lockheed Martin subcontractor that manufactures the landing gear for the F-35 
Lighting Aircraft. The landing gear assembly line recently moved from Independence, 
Ohio, to Fort Worth, Texas. The move began in January 2012, with the first landing gear 
assembly delivered from the new facility in October 2012. In July 2012, UTAS purchased 
the Goodrich division that was the original manufacturer of the F-35 landing gear At the 
time of our assessment, all the engineering offices were still in Ohio; only the assembly 
line and supporting activities had moved to Texas. UTAS provides landing gear for all 
three F-35 variants: CTOL, CV, and STOVL. 

The assessment of UTAS resulted in 51 findings that identified deficiencies in UTAS' 
implementation of the AS9100 Quality Management System. The findings identified 
issues with DCMA delegations and their implementation along with Lockheed Martin's 
insufficiënt supply chain management of critical suppliers. Figure 13 provides UTAS 
findings by AS9100 clause with the following sections summarizing significant issues 
documented during the assessment. 
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Documentation Requirements (4.2) 

We found that the quality assurance inspection control plans used in the inspection 
of hardware were not under configuration control. Four quality assurance inspection 
control plans found on the production floor had been updated by engineering but not 
signed or released in accordance with UTAS procedures. Thirty-seven other documents, 
such as quality assurance documents, landing gear documents, and engineering quality 
assurance procedures; had not been reviewed for adequacy or released in accordance 
with their procedures. 

Human Resources (6.2) 

Competence, Training, and Awareness (6.2.2). We found that UTAS used multiple 
training systems that were not maintained, resulting in incomplete training requirements, 
incomplete records, and the employees' training effectiveness not being assessed. We 
found that training records were incomplete, lacking training requirements and due dates. 
The document system left uncertainty that employees had all the necessary training. 
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Planning of Product Realization (7.1) 

We documented some tools and equipment that had not been proof loaded. Specifically, 
onecranewithvarious riggings was not proof loaded or certified for the F-35 landing gear 
assemblies. This did not meet safety and occupational health requirements. 

Configuration Management (7.1.3). We found that UTAS did not certify or update its 
configuration management policy to reflect the Lockheed Martin requirements. Change 
control, classification of changes, and notification to the customer of pending changes 
were not accomplished. For instance, Lockheed Martin's configuration management 
requirements stated that a change to solder composition should be a major change; 
however, this requirement was not included in the UTAS configuration management plan. 

Customer-Related Processes (7.2) 

Customer Communication (7.2.3). UTAS Fort Worth did not submit minor variances 
to DCMA for review and approval. Lockheed Martin requires that both major and 
minor variances be submitted to DCMA for evaluation to ensure the classification is 
correct and is appropriately dispositioned. We found that minor variances had not been 
submitted; therefore, the classification of nonconformances and dispositions may not be 
technically correct. 

Design and Development (7.3) 

We found that UTAS had not completed the corrosion protection plan for the F-35 
CV landing gear in accordance with the SOW. The corrosion protection plan is necessary 
to define adequate design and maintenance requirements for the landing gear to ensure 
performance in a high humidity and salt spray environment. We also identified that UTAS 
had not received Lockheed Martin's approval for 10 item development specifications 
and an interface control document (ICD) prior to the start of LRIP. Items such as several 
actuators and wire harnesses, a swivel, switch, pressure transducer, fluid sensor, and the 
electrical interface drawing had not been approved. The unapproved item development 
specifications dated back to 2004, and the unapproved electrical ICD dated back to 2009. 

Additionally, we identified that the UTAS testing program did not ensure all technical 
performance requirements were verified. For example, not all Brake/Skid Control 
System performance-based specification requirements were part of the qualification test 
program. In addition, the maintainability and health management performance- 
based requirements had not been demonstrated as required by the performance- 
based specification. 
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Purchasing (7.4) 

UTAS was not flowing down all applicable F-35 Program unique quality assurance 
requirements to its suppliers. UTAS purchase orders did not include Lockheed 
Martin supplier quality assurance requirements, such as quality management system 
requirements, calibration requirements, and Government oversight requirements. 
We also noted numerous deficiencies with UTAS' review and approval of manufacturing 
plans. UTAS did not maintain required monthly supplier quality assurance metrics; 
therefore, supply chain risks were not identified. 

We noted inconsistencies in the information contained in the UTAS SAP" system versus 
the information residing in the supplier quality assurance database. The process 
for supplier rating, certification, and decertification did not follow UTAS procedures 
for procuring parts from qualified and authorized suppliers. Insufficiënt supplier 
management practices may produce defective parts that go undetected. 

Product and Service Provision (7.5) 

Control of Production (7.5.1). We documented that the UTAS master-tooling list 
had not been updated and was inaccurate. The calibration vendor for UTAS identified 
199 UTAS tools in its calibration service purchase order; however, UTAS' master-tooling 
list identified only 128. In addition, many tools on the UTAS production floor including 
the acceptance testing fixtures and numerous small tools and gauges showed no evidence 
of inspection by the quality assurance organization prior to use in production. 

UTAS was not following procedures for FAIs. Specifically UTAS did not ensure that F-35 
landing gear system FAI data packages were consistently reviewed and approved and 
contained the necessary supporting data. FAI is a process used to verify and document 
all essential characteristics on one of the first production components and provides 
confidence that production processes, documentation, and tooiing are sufficiënt to 
begin production. Several FAI packages were categorized as "FAI Not Complete," with no 
corresponding data to explain why they had not been completed. 

Preservation of Product (7.5.5). UTAS procedures for FOD, ESD, and LSLM did not 
ensure product protection; personnel were also not always implementing them correctly. 
In one finding, FOD-controls were not adequate to prevent FOD from being introduced 
into the product. Examples of FOD objects noted in the F-35 assembly area include soda 
bottles on tool/storage racks, a loose protective cap, residual tape, a small plastic bag. 



SAP is a German Corporation that develops business enterprise software. 
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a part identification label, a cigar, and sunflower seeds. We noted that UTAS did not 
have an ESD Workstation or instructions for handling ESD-sensitive components in the 
production area. The UTAS F-35 product Hne contains ESD-sensitive components such as 
the brake control unit and the pressure transducer We also identified several examples of 
inappropriately labeled LSLMs with missing expiration dates. Examples included O-ring 
lubricant, sealing compound, red torque lacquer, and leak detection solution. 

Control of Monitoring and Measurement Equipment (7.6) 

We documented several findings related to the control of monitoring and measurement 
equipment. We found a brake positioning transducer that was calibrated, but the 
inspection report failed to provide the actual calibration values; therefore, it could not 
be determined if the tooi was capable of making reliable measurements. On a transducer 
jig, the flatness/parallelism requirement was not verified as part of the inspection 
procedure and was not recorded in the report. Another finding documented that the 
tooi identification information for torque operations was not recorded on multiple shop 
travelers as required for traceability. Several additional findings documented deficiencies 
with the inventory list, identification methods, and recordkeeping. Although there were 
processes to control monitoring and measurement equipment, the findings indicate that 
improvements are necessary to produce a reliable product. 

Monitoring and IVIeasurement (8.2) 

Internal Audit (8.2.2). The assessment documented one major finding related to internal 
audits. No product audits have been performed on F-35 landing gear assemblies at the 
UTAS Fort Worth facility to ensure hardware meets end-item requirements. Failure to 
complete product audits as planned increases risk of deficiënt processes. 

Control of Nonconforming Product (8.3) 

We found that nonconforming hardware was dispositioned by unauthorized personnel 
instead of the required engineering representative, and did not go through the material 
review process. In addition, one of the nonconforming items we evaluated could not 
be located. 

UTAS was not tracking corrective actions to closure for product nonconformances and 
variances; therefore, UTAS was unaware of the status of these corrective actions. In 
addition, we found that UTAS was not performing root cause and corrective action for 
scrapped F-35 parts and was not submitting variances and scrap dispositions to DCMA as 
required. Insufficiënt corrective action and lack of Government oversight increases the 
risk of additional nonconforming items affecting product reliability and program cost. 
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Lockheed Martin Program Management at UTAS 

We reviewed the effectiveness of Lockheed Martin program management as part of 
our assessment at UTAS. Lockheed Martin flowed down the F-35 Brake/Skid Control 
System performance-based specification with incomplete and conflicting specification 
requirements. We also found at the time of our assessment that Lockheed Martin had not 
funded root cause and corrective action analysis for failed UTAS hardware. Failed parts 
and detailed information of the field failure was not always provided back to UTAS for 
analysis; therefore, UTAS was unable to determine root cause. 

We found that there was no evidence that Lockheed Martin had ever provided 
"concurrence in classification" for minor engineering changes submitted by UTAS. UTAS 
personnel indicated that after 30 days, they assumed that Lockheed Martin agreed with 
the classification. Lack of Lockheed Martin agreement with UTAS-initiated minor changes 
can result in changes being incorporated and implemented without the proper review 
and approval. 

DCMA Oversight 

We found that DCMA Cleveland did not provide the required Government oversight of 
UTAS waivers, deviations, and nonconformances as required by the LOD. Additionally, 
DMCA Cleveland was not delegating nonconforming material review responsibilities to 
the appropriate DCMA offices located at UTAS suppliers. 

UTAS Assessment Summary 

We found a lack of disciplined process controls for FOD, ESD, and LSLM. This could result 
in nonconforming flight hardware or latent defects. 

Several key design and development tasks were not completed. Although the program 
has transitioned to LRIP, UTAS had not received Lockheed Martin's approval for 
10 development specifications and ICDs. UTAS also had notverified technical performance 
requirements testing prior to the start of LRIP. 

UTAS did not conduct internal audits at the Fort Worth facility, and was not tracking 
corrective actions to closure for product nonconformances and variances. The lack of 
an internal auditing program and disciplined corrective action process could result in 
quality assurance issues and additional cost growth to the program. 
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Overall Findings and Recommendations 

The F-35 Program did not sufficiently implement or flow down technical and quality 
management system requirements to prevent the fielding of nonconforming hardware 
and software. This could adversely affect aircraft performance, reliability maintainability 
and ultimately program cost. Lockheed Martin and its subcontractors did not foUow 
disciplined AS9100 Quality Management System practices, as evidenced by 363 findings, 
which contained 719 issues. 



A. Insufficiënt Rigor in Design, Manufacturing, and 
Quality Assurance Processes 

The F-35 JPO, Lockheed Martin, and its subcontractors were not 
ensuring that the necessary quality assurance process controls 
and disciplines were in place to produce a consistent and reliable 
product. This lack of process discipline and attention to detail 
creates an elevated risk of delivering nonconforming aircraft to 
the warfighter. 

The root cause of nearly half of the findings was the lack of adherence to established 
processes and procedures or insufficiënt detail in documentation. The major process 
discipline issues were in the foUowing areas. 

• Planning for Product Realization - Production planning lacked the 
appropriate level of detail, design changes were uncontrolled or unapproved, 
and system-level plans were not maintained. As a result, documentation 
did not always match the processes and equipment required to perform the 
manufacturing operation. 

• Design and Development - Contractor personnel were using insufficiënt or 
unapproved test plans and procedures. Systems engineering technical review 
entrance and/or exit criteria were not met. The criteria ensure the program 
is ready to proceed to the next production milestone or test event. 

• Software Development - In some cases, software design and development 
activities lacked definition and rigorous adherence to documented 
procedures and processes, resulting in software development requirements 
not implemented, traceable, and verified. 
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• Manufacturing Operations - Contractor personnel were not following 
documented manufacturing and assembly process instructions necessary 
to ensure conformance to end-item requirements, product protection, and 
material control. 

• Production Tooiing - Uncontrolled and unproven tooiing was in use on the 

production floor, resulting in product nonconformances. 

• Calibration of Measurement Systems - Multiple sites had inadequate 
calibration management systems. Contractors were not performing out- 
of-tolerance condition analyses, using sufficiënt calibration procedures, 
and removing expired calibrated equipment from use. Accurate test and 
measurement equipment is necessary to provide accurate measurements that 
determine product acceptability. 

• Engineering Change Management - Lockheed Martin was not reviewing 
and approving all subcontractor submittals for variances and engineering 
change requests for which Lockheed Martin is the sole design authority. 
Additionally, Lockheed Martin was not providing concurrence for engineering 
change classifications for subcontractor engineering changes. 

Recommendation A— Management Comments and 
Our Response 

We recommend that the Joint Program Office: 

1. Ensure compliance with AS9100 throughout the F-35 supply chain. 

Joint Program Office Comments 
JPO agreed and stated: 

AS9100 [Quality Management Systems - Requirements for 
Aviation, Space, and Defense Organizations] is an 
F-35 contractual requirement. The prime contractor 
[Lockheed Martin (LM]) is AS9100 certified and has flowed 
down this quality requirement to its supply chain. AS9100 
certification is granted by a third party accredited entity. 
Periodic reviews are held by that third party entity to maintain 
certification. Annual self audits are conducted by both the 
prime contractor and its suppliers to monitor compliance 
within their respective facilities. The Government has had 
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limited insight to results of these self audits, however, DCMA 
performs onsite process reviews as part of its oversight 
activity. DCMA at Lockheed Martin Fort Worth reorganized in 
February 2013 to provide more focus and inspection 
emphasis to ensure AS9100 compliance and a quality 
product. Additionally, JPO reviews AS9100 certification 
status of critical suppliers during their annual Production 
Readiness Reviews [PRR). These reviews serve as a LRIP 
risk management tooi conducted with the prime contractor 
and selected major critical suppliers, covering Technical, 
Production and Sustainment requirements. Suppliers must 
demonstrate consistent adherence of product compliant to 
contractual [AS9100] and applicable statutory and regulatory 
requirements. Findings of this IG audit have caused both 
contractor and Government entities to adjust the frequency 
and scope of independent audits to ensure greater insight to 
AS9100 compliance. 

Our Response 

The comments from JPO are responsive, and the actions taken appear to meet the 
intent of the recommendation. 

2. Ensure that Lockheed Martin approves all design and material review 
board changes and variances with Government concurrence. 

Joint Program Office Comments 
JPO agreed and stated: 

Current F-35 Program plans [such as the Configuration 
Management Plan) require that all major design changes 
and variances for production aircraft receive Government 
concurrence. The Government maintains limited insight 
of the contractor's Material Review Board [MRB) process. 
Subsequently the Government has reprioritized and refined its 
engagement in that process to ensure improved effectiveness. 
For nonconformances that require MRB disposition, JPO 
delegates the approval of Class II [minor] changes to DCMA 
and reviews all Class I changes that affect form, fit, or 
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function. A Class I change request is dispositioned through 
the JPO Change Review Board (JCRB), which performs a 
technical and business assessment and provides a written 
recommendation to the Joint Configuration Control Board 
[JCCB). With the input of major stakeholders, the JCCB 
adjudicates the recommended change and provides that 
decision to Contracts, who notifies the Prime Contractor to 
proceed in incorporating the change. 

Any subcontractor submittal for variances and engineering 
change requests, for which Lockheed Martin is the sole 
design authority, must be go through Lockheed Martin's 
change control boards for review and approval. Furthermore, 
Lockheed Martin must provide concurrence for engineering 
changes relating to subcontractor owned designs. Various 
configuration management checks and balances [such as 
Physical and Functional Configuration Audits) are in place to 
ensure control. 

Our Response 

The comments from JPO are responsive. The actions taken appear to meet the intent 
of the recommendation; however, the DoD OIG suggests that the JPO and 
Lockheed Martin maintain cognizance of class II changes. 

3. Perform process proofing of all critical processes to include first 
article inspections. 

Joint Program Office Comments 
JPO partially agreed and stated: 

The Government does not have the resources to perform 
process proofing of all critical processes. Responsibility and 
accountability rests on the prime contractor, with oversight 
from the Government. Process proofing and First Article 
Inspections are Program Plan and contractual requirements. 
The Government performs audits of process compliance to 
ensure the contractor is controUing critical operations. The 
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Government closely monitors scrap, rework, and repair to 
ensure the contractor is conducting proper root cause analyses 
to implement appropriate corrective actions. The Government 
also monitors the quality of delivered product to ensure control 
of escaped defects. 

JPO requires the contractor to perform First Article Inspections 
[FAI] and process proofing as part of their implementation 
of AS9100 and the F-35 Program Quality Management Plan. 
Over 27,500 original baseline FAIs have been completed and 
approximately 400 more delta FAIs will be performed for 
parts due to concurrency, producibility, process changes, 
etc. In addition, the prime contractor has implemented 
variation management, which is an advanced quality system 
technique that focuses on defect prevention and continuous 
improvement through the identification of key product and 
process characteristics. This allows greater control of the 
manufacturing process. 

Our Response 

Although we understand the concern for Government resources, we still recommend 
that JPO ensure process proofing of all critical processes. A widely accepted industry 
handbookon quality, /uran's Quality Handbook, defines critical properties as: "Critical 
properties are those properties for which the made-product total variability is so 
wide as to cause persistent difficulty in meeting current or near-term anticipated 
customer needs." The Government must ensure that the contractor performs some 
level of process proofing to prevent, as Juran's Quality Handbook states, "variability 
in critical product characteristics, which is the enemy of quality of conformance." 

As evidenced by the findings, process proofing had not been accomplished to the 
level to ensure product repeatability. It is the DoD OIG's position that without 
process proofing of all critical processes, quality assurance may not be 
achieved. Therefore, we request further comments from the JPO in response to 
the final report. 

4. Modify its contracts to include a quality escape clause to ensure the 
Government does not pay for nonconforming product. 
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Joint Program Office Comments 
JPO agreed and stated: 

JPO concurs that greater incentives should be taken to 
preclude the cost of poor quality. Major non-conformances 
that do not meet engineering specification are identified 
and adjudicated for acceptance or rejection with 
considerations/withholds placed on a variance. Minor 
non-conformances are reworked, repaired, or used as is. 
The program has transitioned to fixed price type contacts 
with a 0/100 share ratio and clauses will prescribe 
to applicable FAR [Federal Acquisition Regulation] 
requirements. Along with considerations/withholds on 
non-conformances, this contract structure will facilitate 
greater incentive by the contractor to provide quality 
assurance in order to maintain cost and schedule obligations 
without over running their negotiated budget. The 
contractor will absorb a level of cost for poor quality. In 
addition, the JPO continues to establish quality performance 
targets with commitments from JPO and LM senior 
leadership. These targets are based upon continuous 
learning in fabrication and assembly operations, corrective 
action implementation, in addition to product and 
process improvements as a result of affordability and 
concurrency changes. 

Our Response 

The comments from JPO are responsive, and the actions taken appear to meet the 
intent of the recommendation. 

5. Assess the impacts and risks to all delivered aircraft for all findings. 

joint Program Office Comments 
JPO agreed and stated: 

The impacts and risks of all findings were assessed as 
part of the Corrective Action Request (CAR)/Corrective 
Action Plan (CAP] process. When a CAR was written, the 
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F-35 fleet was assessed for impact. Concurrence on each CAP 
considered effectivity, safety, and contract elements. This 
determined priority and timing required to implement the 
corrective action into the affected aircraft as appropriate. 

Our Response 

The comments from JPO are responsive, and the actions taken appear to meet the intent 
of the recommendation. 

B. Critical Safety Item Program Not Implemented 

The F-35 JPO did not include CSI requirements on the Lockheed 
Martin prime contract as required by Public Law and DoD 
policy. Aircraft CSIs were not identified and critical parts were 
not manufactured with CSI controls and verifications. Delivered 
aircraft may pose an increased safety of flight risk due to the lack 
of critical process control. 

JPO has not flowed down a contractual requirement to Lockheed Martin (and 
subsequently its subtier suppliers) to implement a CSI management program in 
accordance with Public Law and DoD policy. A CSI is a part, assembly or support 
equipment whose failure could cause loss of life, permanent disability or major 
injury, loss of a system, or significant equipment damage. Special attention should be 
paid to CSIs to prevent the potential catastrophic or critical consequences of failure. 
CSIs require special handling, engineering, manufacturing, and inspection 
documentation to control and ensure safety of flight. 

Public Law 108-136, Section 802, "Quality control in procurement of aviation CSIs and 
related services," requires DoD to prescribe a quality control policy for the procurement 
of aviation CSIs. Joint Service CSI Instruction, "Management of Aviation Critical Safety 
Items," implements the DoD CSI program and establishes the policies, procedures, 
and responsibilities to manage CSI. The Joint Aeronautical Commanders Group, "JACG 
Aviation Critical Safety Item Handbook," implements the policies in the Joint Service CSI 
Instruction and describes the technical and quality assurance requirements for a Prime 
Manufacturer CSI program. Without a CSI program, there is an increased safety risk to the 
aircraft and warfighter. 
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Recommendation B— Management Comments and 
Our Response 

We recommend that the Joint Program Office: 

1. Implement an aviation critical safety item program that meets the 
requirements of Public Law and DoD policy, which would include 
flow down of requirements for a critical safety item program to 
Lockheed Martin and its subcontractors. 

Joint Program Office Comments 
JPO agreed and stated: 

JPO is working with the prime contractor in developing 
a Request for Proposal [RFP] for a two-phased CSI 
Non Recurring Engineering [NRE) approach. This will require 
the development of comprehensive CSI requirements which 
will be added to the SDD contract's Statement of Work. These 
requirements will address the Identification of critical 
characteristics that enable compliance at the prime contractor 
and throughout its supply base. Once the SDD NRE efforts 
are completed, the fuU scope of the F-35 CSI program will 
be included in future production and sustainment contracts. 
Commencement of certain initial recurring tasks will be 
implemented using the LRIP 6 contract. In the interim, the 
contractors' integrity and quality programs are providing 
assurance for continued aviation safety. 

Our Response 

The comments from JPO are responsive, and the actions appear to meet the intent of 
the recommendation. 

2. Assess the impacts and risks to all delivered aircraft for critical safety 
item deficiencies. 
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Joint Program Office Comments 
JPO agreed and stated: 

Deficiencies and impacts to critical parts are identified and 
assessed during quality assurance practices and testing 
operations as part of the SDD design and product verification 
processes. If safety issues and limitations are discovered, 
changes are initiated and incorporated as warranted to the test 
and production aircraft. Initial and continued airworthiness 
is a major focus of the F-35 Class Desk, Airworthiness Team, 
Integrated Test Team, and Integrated Product Teams. The 
SDD test aircraft have more than 5,000 flight hours. The 
combined F-35 fleet has accumulated more than 7,000 flight 
hours. This is not to say that there are no significant issues or 
discoveries. Rather, there is an upfront understanding whether 
any new discovery poses an increased safety risk. If necessary, 
proper actions are taken, such as suspending flight operations, 
directing inspections, adding flight limitations or restrictions, 
and directing hardware or software modifications to mitigate 
the safety risks. 

Our Response 

The comments from JPO do not meet the intent of the recommendation. 
Per our findings the JPO did not create a CSI program in accordance with 
Public Law 108-136, Section 802. The DoD OIG desires the Identification 
and confirmation from the Engineering Support Activity that all safety risks for 
aircraft manufactured and delivered to the Government using the methodology 
identified in the CSI handbook is complete. Therefore, we request further 
comments from the JPO in response to the final report. 

C. Requirements Not Flowed Down 

JPO and Lockheed Martin did not ensure that all essential quality 
assurance and technical requirements were flowed down to 
the supply chain. This resulted in suppliers providing critical 
F-35 products that may not meet intended performance and 
reliability requirements. 
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We noted a significant number of findings in this area. These include the following. 

• First Article Inspection - Lockheed Martin flowed down conflicting 
requirements for FAIs to its suppliers. The lack of definitive FAI 
requirements was affecting the suppliers' ability to qualify its processes 
and ensure delivery of compliant product. 

• Configuration Control - Lack of flow down and communication between 
Lockheed Martin and suppliers was causing unclear configuration 
management and status accounting requirements. Configuration issues at 
suppliers were resulting in products with unknown configurations. 

• Design and Development Requirements - Lockheed Martin did not flow 
down all the necessary design and development requirements and ensure 
requirements were verified and traceable. This impacted the suppliers' ability 
to develop products that meet expected performance requirements. 

• Diminishing Manufacturing Supply and Materiel Shortages - JPO had not 

funded Lockheed Martin to implement a DMSMS program. As a result, the 
program was not proactively forecasting for part obsolescence. 

Recommendation C— Management Comments and Our 
Response 

We recommend that the Joint Program Office perform technical and quality 
assurance requirement flow down and verification throughout the F- 3 5 
supply chain. 

Joint Program Office Comments 
JPO partially agreed and stated:: 

The Government does not have the responsibility or resources 
to perform requirement flow down verification throughout the 
prime contractor's entire supply chain. The prime contractor 
is responsible and accountable, with oversight from the 
Government. The Government performs spot checks and 
conducts surveillance to ensure the contractor is flowing 
down requirements to its subcontractors. The Government 
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monitors this closely through participation in Design Reviews, 
Production Readiness Reviews, Structural Equivalency 
Reviews, Program Management Reviews, production floor 
inspections, configuration audits, etc. On site surveillance is 
primarily performed by DCMA to ensure the contractor and its 
subcontractors are complying with contractual requirements. 

Our Response 

The DoD OIG does not agree that the Government does not have the responsibility 
to ensure that the prime contractor and the supply chain meet requirements. In 
accordance with DoDD 5000.01, section 3.5, the Program Manager has the responsibility 
for and authority to accomplish program objectives for development, production, and 
sustainment to meet the user's operational needs. The intent of our recommendation 
is for JPO to ensure that technical and quality assurance requirement flow down occurs 
by estabUshing the appropriate contractual requirements; such as having the prime 
contractor create a database showing each supplier and all appücable performance 
and quality assurance requirements. Contractual deliverables such as database access 
and various monthly metrics would provide the Government additional confidence 
that this has occurred. The lack of a requirement flow down process to date has 
hampered the program's understanding of supplier conformance to performance and 
quality contractual requirements. Therefore, we request further comments from JPO in 
response to the final report. 

D. Ineffective Quality Assurance Organization 

JPO's quality assurance organization did not have the appropriate 
resources and authority to effectively manage DoD's largest 
acquisition program. The lack of a strong and effective quality 
assurance organization contributed to the program's cost, 
Schedule, and performance issues. 

DoDD 5000.01, "The Defense Acquisition System," designates authority and responsibility 
to the program manager within JPO to accomplish program objectives for development, 
production, and sustainment to ensure that reliable systems are delivered to the 
warfighten DoDD 5000.01 goes on to state that the program manager, "shall be the single 
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point of accountability for accomplishing program objectives for total life-cycle systems 
management." As evidenced by our assessment that identified 363 findings, JPO appeared 
to rely on Lockheed Martin and DCMA to identify, report, and address quality assurance 
issues. This indicates a lack of quality assurance and technical expertise within JPO to 
recognize F-35 supply chain issues. 

Furthermore, the Lockheed Martin Aeronautics quality assurance leadership team 
reports to the Lockheed Martin Aeronautics operations leadership team and does not 
have the organizational independence or inherent authority to enforce quality assurance 
internally and throughout the F-35 supply chain. It should be noted that Lockheed Martin 
Space Division has an independent quality assurance organization that does not report 
to operations. 

Recommendation D— Management Comments and 
Our Response 

We recommend that the Joint Program Office establish an independent quality 
assurance organization, which has the authority and resources to enforce the 
AS9100 Standard and F-35 product quality. 

Joint Program Office Comments 

JPO disagreed with the recommendation. JPO stated: 

JPO acknowledges opportunities to improve management of 
Quality performance and the benefits of realignment within 
the F-35 Program organizational structure to better support 
transition from Low Rate Initial Production to Full Rate 
Production. These efforts have commenced to include an 
increase in resources and realignment of the Quality Team 
to report to program leadership. The JPO non-concurs with 
establishing an independent quality assurance organization. 
DCMA performs the role of the independent quality 
assurance organization for the F-35 and other DoD programs. 
DCMA conducts process surveillance against the AS9100 
Standard and enforces F-35 product quality requirements. 
Additionally, the JPO Quality Team is staffed from two 
independent organizations; Air Force Life Cycle Management 
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Center (AFLCMC) and Naval Air Systems Command (NAVAIR). 
While serving on the JPO Quality Team, these individuals 
retain their responsibility to invoke their independent 
AFLMC and NAVAIR leadership in the event the JPO does not 
properly address quality concerns. 

Our Response 

We disagree with JPO's approach because DCMA, AFLCMC, and NAVAIR are not 
accountable for program quality assurance goals. In accordance with DoDD 5000.01, 
section 3.5, the Program Manager has the responsibility for and authority to 
accomplish program objectives for development, production, and sustainment to 
meet the user's operational needs. Quality assurance is inherently a Government 
program office's responsibility. As evidenced by our findings, JPO did not enforce 
F-35 product quality requirements. DCMA uses a "riskbased surveillance approach" 
to apply resources across the DoD. Based on the findings from our assessment, DCMA 
did not apply the appropriate resource throughout the supply chain. JPO must ensure 
that Government resources are integrated into the program, efficiently applied, and 
understand the system criticality of each aircraft component. It should also ensure 
that design, test, and manufacturing issues are addressed with long-term, permanent 
Solutions. The DoD OIG seeks an effective quality assurance organization that meets 
these expectations. Therefore, we request further comments from JPO in response to 
the final report. 

E. Ineffective DCMA Oversight 

DCMA was not performing quality assurance oversight 
commensurate with product criticality. Insufficiënt written 
direction from JPO coupled with inadequate execution from 
DCMA resulted in ineffective Government oversight of the 
F-35 Program. 

In accordance with the F-35 Program memorandum of agreement, DCMA has 
been delegated the responsibly of final acceptance of each production article. The 
memorandum states "DCMA will issue LODs to support Contract Management Offices for 
key JSF subcontractors as defined by DCMA and based on the analysis of subcontractor 
performance data and risk." In addition, DCMA will analyze contractor-earned value 
data and provide an assessment of program risk based on cost, schedule, and technical 
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performance. DCMA will also maintain the Joint Surveillance Plan [between DCMA and 
Lockheed Martin]. The plan provides the details for accomplishing system surveillance 
and maintenance. 

Although there was a memorandum of agreement between JPO and DCMA, it was 
inadequate for the complexity of the F-35 Program. The memorandum of agreement 
did not provide any specifics to ensure adequate oversight of the F-35 supply chain. 
DCMA Fort Worth did not issue letters of delegation to cognizant DCMA offices at key 
F-35 suppüers based on the analysis of supplier performance data and risk. DCMA did 
not consistently: 

• develop or implement risk-based surveillance plans in accordance with 
DCMApoücies, 

• participate in material review board activities, 

• adequately conduct oversight activities, and 

• identify and report oversight activities performed at lower tier suppliers. 

Recommendation E— Management Comments and 
Our Response 

1. We recommend that the Joint Program Office: 

a. Revise the Defense Contract Management Agency memorandum of 
agreement to provide explicit surveillance criteria for mission-critical 
hardware and software, to include, hut not limited to, material review, 
mandatory government inspection, process audit, product acceptance. 

Joint Program Office Comments 
JPO agreed and stated: 

The Memorandum of Agreement (MOA], annex A-P, dated 
February 2011, between the JPO and DCMA is being updated 
to reflect Program objectives requiring DCMA support in the 
foUowing areas: Management of contractor Quality Process 
implementation, control, maturity, and verification; Material 
Review/Change Board Process authority and related task 
requirements, Foreign Object Damage [FOD] management, 
Government Flight Representative surveillance, Safety of 



68 I DODIG-2013-140 



Overall Findings and Recommendations 



Flight inspection requirements, contractual Corrective Action 
Request notification, Air system acceptance [DD250), Supply 
Chain Management, etc. However specific surveillance 
activities and priorities are articulated via other tools, such 
as Quality Assurance Letters of Inspection [QALI's]. 

It is important to note that the MOA update, though crucial in 
shaping organizational roles and responsibilities that address 
the findings in this report, is only part of the corrective action 
process being implemented. Execution of DCMA policies, 
enhanced communication between the JPO and DCMA, and 
enhanced inter-DCMA Communications with supplier DCMA 
organizations all play a role in improving surveillance. In 
addition to the MOA, DCMA utilizes risk based surveillance 
plans to support F-35 contracts. As part of the functional 
surveillance planning process, DCMA determines what types 
of surveillance techniques will be applied to ensure a supplier 
has adequate quality processes in place and meet all contractua 
Irequirements. Risks are identified through historical 
knowledge of the suppliers systems and processes, results of 
surveillance execution, analysis of DCMA internal data, supplier 
data and customer data. 

Our Response 

The comments from JPO are responsive and the actions met the intent of 
the recommendation. 

b. Ensure that Defense Contract Management Agency is performing quality 
assurance oversight commensurate with product criticality. 

Joint Program Office Comments 
JPO agreed and stated: 

DCMA has been, and continues to be, responsive to product 
criticality while performing its contact administration 
quality assurance functions. This includes optimizing 
quality assurance oversight of contractors' critical processes, 
operations, and product. Product criticality is governed by 
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Federal Acquisition Regulation derived surveillance strategies. 
FAR/DFAR regulatory requirements as manifest in F-35 
contracts are also guided by DCMA policies. DCMA policies 
define risk considerations and thresholds in developing 
surveillance plans. Quality specialists determine what types of 
surveillance techniques will be applied to ensure a supplier 
has adequate quality processes in place. Surveillance activities 
and data analysis allow DCMA to re-evaluate risk levels and 
where appropriate adjust surveillance plans. 

As non-conformance is identified, JPO may request special 
surveillance be performed in other areas due to severity 
or whether they are systematic in nature. This is captured 
in other documents such as QALI's. The DCMA monthly 
report also is a means of communicating quality assurance 
oversight as it pertains to production status. Continuous 
communication between JPO and DCMA to discuss and resolve 
quality issues is critical to meeting Program objectives. 
Performing quality assurance in accordance with product 
criticality is central to how DCMA conducts contract surveillance. 

Our Response 

The comments from JPO do not meet the recommendation intent. As evidenced by 
the findings, DCMA is not providing sufficiënt supply chain oversight commensurate 
with product criticality. The term "surveillance," as defined in Federal Acquisition 
Regulation (FAR) 42.11, is broad and does not specifically equate to government 
verification or inspection of manufacturing processes or products. As an example, 
the DCMA surveillance for a supplier could be nothing more than an itinerant DCMA 
person visiting a particular supplier every six months with the visit entailing a brief 
walk through of the facility. Therefore, it is our position that JPO's quality assurance 
organization should make the final decision on supplier risk, and the type and amount 
of DCMA surveillance. We request further comments from the JPO in response to the 
final report. 
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2. We recommend that the Defense Contract Management Agency: 

a. Provide a comprehensive quality assurance oversight plan for 
Joint Program Office approval to be included in the memorandum 
of agreement. 

Defense Contract Management Agency Comments 
DCMA partially agreed. DCMA stated: 

DCMA will ensure the MOA, as currently being updated, 
complements DCMA risk based surveillance planning. The 
DMCA surveillance planning process is risk based and 
developed through historical knowledge of the suppliers 
systems and processes, results of surveillance execution, 
analysis of DCMA internal data, supplier data and 
customer data. 

DCMA is also implementing a dedicated Supply Chain 
Management team at LMFW [Lockheed Martin, Fort Worth] 
to address the significant supply chain oversight challenge of 
the F-35 program. The focus of the team will be to evaluate 
LMFWs supply chain processes and system as well as a 
more focused emphasis on risk suppliers throughout the 
Supply Chain. 

Our Response 

The comments from DCMA do not meet the recommendation intent. As stipulated 
in our E.l.b response, it is important for JPO's quality assurance organization 
to make the final decision on supplier risk and the type and amount of DCMA 
oversight. A comprehensive quality assurance oversight plan would specifically 
identify DCMA oversight for critical assemblies: inspection, verification, and 
process proofing. Therefore, we request further comments from the DCMA in 
response to the final report. 

b. Audit the execution of the quality assurance oversight plan throughout 
the F-35 supply chain. 
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Defense Contract Management Ag ency Comments 
DCMA agreed and stated: 

DCMA will comply with its quality assurance oversight plan. 
Regular audits of supplier and prime DCMA organizations 
are crucial to maximizing the effectiveness of risk based 
surveillance. Within DCMA, Management Review Teams (MRT) 
provide comprehensive functional reviews on our quality 
program. These risk-based audit efforts, are conducted by 
independent quality experts and occur throughout the entire 
DCMA organization ensuring coverage of the F-35 supply chain. 
Continued communication of audit results with our customers 
and enhanced communication between DCMA supplier CMOs 
will also supplement a robust audit process. 

Our Response 

The DCMA comments do not meet the recommendation intent. DCMA states they 
will comply with its existing quality assurance oversight plan, which does not 
appear to be the comprehensive F-35 quality assurance oversight plan sought in our 
E.2.a recommendation. In addition, the DCMA audit and MRT process cited is not 
new and as identified by our assessment was insufficiënt. Therefore, we request 
further comments from the DCMA in response to the final report. 
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Appendix A 

Scope and Methodology 

We conducted this assessment from February 2012, through July 2013, in accordance 
with the Council of the Inspectors General on Integrity and Efficiency, "Quahty Standards 
for Inspection and Evaluation." Those standards require that we plan and perform the 
assessment to obtain sufficiënt, appropriate evidence to provide a reasonable basis for 
our findings and conclusions based on our assessment objectives. We believe that the 
evidence obtained provides a reasonable basis for our findings and conclusions based on 
our assessment objectives. 

To review the quality management system, our assessments focused on the following: 

• Applicable Statutory/regulatory requirements 

• Contractual quality management system (AS9100] 

• Contractual quantity clauses 

• Internal quality assurance processes and procedures 

• Software quality assurance process 

• Aviation CSI 

Use of Technical Assistance 

We used assistance from quality assurance engineers and quality assurance specialists 
with a background in defense and aerospace systems. We established teams of subject 
matter experts who assessed to the AS9100C Quality Management System Standard. 
The subject matter expert teams consisted of 14 to 18 quality assurance engineers, 
trained and certified in AS9100, who had an average of 15 years of quality assurance, 
audit experience. Additionally our teams included subject matter experts in military 
aviation CSI. 
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Prior Coverage 

During the last 5 years, the Government Accountabüity Office (GAO) and the 
Department of Defense Inspector General [DoD IG] issued 26 reports discussing 
F-35 Joint Strike Fighten Unrestricted GAO reports can be accessed over the 
Internet at http : / / www.gao.gov . Unrestricted DoD IG reports can be accessed at 
http : / / www.dodig.mil/pubs/ index.cfm . 

GAO 

Report No. GAO-13-690T, "F-35 Joint Strike Fighter: Restructuring Has Improved the 
Program, but Affordability Challenges and Other Risks Remain," June 19, 2013 

Report No. GA0-13-294SP, "Defense Acquisitions: Assessments of Selected Weapon 
Programs," March 28, 2013 

Report No. GAO-13-309, "F-35 Joint Strike Fighter: Current Outlook Is Improved, but 
Long-Term Affordability Is a Major Concern," March 11, 2013 

Report No. GAO-12-437, "Joint Strike Fighter: DOD Actions Needed to Further Enhance 
Restructuring and Address Affordability Risks," June 14, 2012 

Report No. GAO-12-400SP, "Defense Acquisitions: Assessments of Selected Weapon 
Programs," March 29, 2012 

Report No. GAO-12-525T, "Joint Strike Fighter: Restructuring Added Resources and 
Reduced Risk, but Concurrency Is Still a Major Concern," March 20, 2012 

Report No. GAO-11-903R, "Joint Strike Fighter: Implications of Program Restructuring 
and Other Recent Developments on Key Aspects of DOD's Prior Alternate Engine 
Analyses," September 14, 2011 

Report No. GA0-11-677T, "Joint Strike Fighter: Restructuring Places Program on Firmer 
Footing, but Progress Is Still Lagging," May 19, 2011 

Report No. GAO-11-325, "Joint Strike Fighter: Restructuring Places Program on Firmer 
Footing, but Progress Still Lags," April 7, 2011 
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Report No. GAO-11-450T, "Joint Strike Fighter: Restructuring Should Improve Outcomes, 
but Progress Is Still Lagging Overall," March 15, 2011 

Report No. GAO-11-323R, "Tactical Aircraft: Air Force Fighter Force Structure Reports 
Generally Addressed Congressional Mandates, but Reflected Dated Plans and Guidance, 
and Limited Analyses," February 24, 2011 

Report No. GA0-11-171R, "Defense Management: DOD Needs to Monitor and Assess 
Corrective Actions Resulting from lts Corrosion Study of the F-35 Joint Strike Fighter," 
December 16, 2010 

Report No. GAO 10-1020R, "Joint Strike Fighter: Assessment of DOD's Funding Projection 
for the F136 Alternate Engine," September 15, 2010 

Report No. GAO-10-789, "Tactical Aircraft: DOD's Ability to Meet Future Requirements 
is Uncertain, with Key Analyses Needed to Inform Upcoming Investment Decisions," 
July29, 2010 

Report No. GAO-10-388SP, "Defense Acquisitions: Assessments of Selected Weapon 
Programs," March 30, 2010 

Report No. GAO-10-478T, "Joint Strike Fighter: Significant Challenges and Decisions 
Ahead," March 24,2010 

Report No. GAO-10-382, "Joint Strike Fighter: Additional Costs and Delays Risk Not 
Meeting Warfighter Requirements on Time," March 19, 2010 

Report No. GAO-10-520T, "Joint Strike Fighter: Significant Challenges Remain as DOD 
Restructures Program," March 11, 2010 

Report No. GAO-09-711T, "Joint Strike Fighter: Strong Risk Management Essential as 
Program Enters Most Challenging Phase," May 20, 2009 

Report No. GAO-09-326SP, "Defense Acquisitions: Assessments of Selected Weapon 
Programs," March 30, 2009 

Report No. GAO-09-303, "Joint Strike Fighter: Accelerating Procurement before 
Completing Development Increases the Government's Financial Risk," March 12, 2009 
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Report No. GAO-08-782T, "Defense Acquisitions: Better Weapon Program Outcomes 
Require Discipline, Accountability, and Fundamental Changes in the Acquisition 
Environment," June 3, 2008 

Report No. GAO-08-467SP, "Defense Acquisitions: Assessments of Selected Weapon 
Programs," March 31, 2008 

Report No. GAO-08-569T, "Joint Strike Fighter: Impact of Recent Decisions on Program 
Risks," March 11, 2008 

Report No. GAO-08-388, "Joint Strike Fighter: Recent Decisions by DOD Add to Program 
Risks," March 11, 2008 

DoD IG 

Report No. DODIG-2013-031, "Audit of the F-35 Lightning II Autonomie Logistics 
Information System (ALIS)," December 10, 2012 
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Notices of Concern and JPO Responses 
NOC Lockheed Martin 



INSPECTOR GENERAL 

DEPARTMENT OF DEFENSE 

4800 MARK CENTER DRIVE 
ALEXANDRIA, VIRGINIA 22350-1500 

APR 25 2012 

MEMORANDUM FOR PROGRAM EXECUTIVE OFFICE JOINT STRIKE FIGHTER 

SUBJECT: Notice of Concern-Quality Assurance Assessment of the F-35 Program 
(Project No. D2012-DT0TAD-0003) 

We are issuing this Notice of Concern (NOC) to inform you that the Department of 
Defense, Inspector General (DoD IG) team identified issues that require your attention. During 
the Quality Assurance assessment' of the F-35 program at Lockheed Martin Aeronautics (LMA), 
Fort Worth, Texas, from March 26, 2012 to April 6, 2012, the team identified 70 findings, 29 of 
which we consider to be major (systemic) and 41 to be minor. 

We sorted the findings into six broad categories: Document Control, Risk Management, 
Process Discipline, Process Proofing, Foreign Object Debris (FOD), and Critical Safety Items 
(CSI). Our assessment of each of these areas is as foUows: 

• Document and configuration control is weak and needs management attention. 

• Risk management needs to be handled upfront programmatically and technically. 

• Process discipline is lacking in many areas reviewed by the team. 

• Production processes have not been thoroughly proofed. 

• FOD discipline is lacking and requires major cultural changes. 

• CSI Development needs immediate and complete implementation. 

On average, at final assembly each aircraft has 200+ corrective actions requiring rework 
or repair. The DoD IG team's overall conclusion is that LMA's, Fort Worth, Texas quality 
management system and the integrity of the F-35 product are jeopardized by a lack of attention 
to detail, inadequate process discipline, and a "we will catch it later" culture. We believe the 
quality assurance culture at LMA, Fort Worth, Texas must improve and that robust technical 
oversight by the govemment is required to ensure program performance and mission success. 

Our findings are attached to this memorandum tbr your review and comment. 
DoD Directive 7650.3 requires that recommendations be resolved promptly. Please provide 
comments that state whether you agree or disagree with the findings and recommendations. If 
you agree with our recommendations, describe what actions you have taken or plan to take to 
accomplish the recommendations and include the completion dates of your actions. If you 
disagree with the recommendations or any part of them, please give specific reasons why you 
disagree and propose altemative action if appropriate. Once we complete our five assessments, 



' The assessment was conducted in accordance with the Council of the Inspector General on 
Integrity and Efficiency (CIGIE) Quality Standards for Inspection and Evaluation and guidance 
in AS9101, Quality Management Systems Audit Requirements for Aviation, Space, and Defense 
Organizations. 
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NOC Lockheed Martin (cont'd) 



we will provide a report on oiir fitidiiigs, along witii any additioiial inafters that miglit come to 
om attcntion. 

Wi; sliüiild iL'ucivi; yoilr coimututs by May 25. 2012. Wc uormally iiichldc topics of llic 
coiimieuts iu the report. If yoii cousider auy matters to Lie exempt fiom public release, you 
sliüuld mark llicm clcHily fui Iiispcclor Gcult»! cuusidumtiuii. 

IfDossible^end a portable doeumeiit lortnat (pdf) file eontainiug your comments to 
Copies uf ytnii coimneuls must have llie atliial sigiiahire of the 
aiitliorizmg otlicial tor your orgaiiizatioii. We aie iiuable to accept the 'Sigued/ sjTuboI iu place 
uf Ibe acliial siiaiatui^. If you airaujje lo seud classiEed coumieuts elettruuicallv, voii miisl seud 
tbem over the SECRET lutemet Protocol Router Network (SIPRKCT). 

preciate the coiirtesies extended to the sta ff. Please diieet questions ^^^^^^^^1 




Raudolpb R. Stoue, SES 
Duputy lüHpcului Gcucral 
Policy and Oversight 



Attacbiueuts: 
As slalcd 




78 I DODIG-2013-140 



Appendixes 



JPO Response to NOC Lockheed Martin 




JOINT STRIKK FIGHTER PR0(;RAM 
m 12™ Streel South, Suite 600 
Artington, Virginia 22202'5402 




From: Program Exociitive Officer, F-35 Lightning II 

Tq,j Deputy Inspector Ganeral, Policy and Oversight 2 3 

Department of Defense Inspector General 

Subj: RESPONSE, MOTICE OF CONCERN - QUALITY ASSURAWCE 
ASSESSKENT OF THE F-35 PROGRAM 

1. I receivetj and analyzed your 25 April 2012 Notice of 
Concem-Quality Assurance Assessment of the Program 
(Project No. D2012-DT0TAD-0D03 ) regarding your visit to Lockheed 
Martin Aeronautics (LMA) , Fort Worth, Texas from 26 March to 6 
April 2012- I appreciate your team's efforts to assess 
conformity to F-3S quality management systems, contractual 
(ïuality clauses, and intemal processos and procedures, to 
include Aviation Critical Safety Items. I view your assessment 
as B vaJnatile rooi to independent! y evaluare tTie areas for 
itnprovement within the F-35 program, with che ultimate goal of 
producing a guality, timely, and cost-ef f ective weapon syatera 
for the Onited States and our allies. 

2. The P-35 Joint Program Office (JPO), in concert with the 
Defense Contract Management Agency (dcma) and lma, reviewed eacb 
of the assessment ' 3 findings and recommendations . I accept all 
of the findings issued relative to LMA as valid. I agree Chat 
all of the recotnmendations issued relative to LMA should be 
assessed and implemented to the ma>:imuin practicable extent. 
Additionally, the F-35 JPO reviewed the findings and 
reoommendations issued relative to the F-35 JPO, and I also 
accept all of these findings as valid. I accept all of the 
recoinmendations issued relative to the F-35 JPO. A detailed 
discussion of our path forward is contained at ^pendix A and is 
wartted for Inspector General Consideration.. 

3. We will continue to work with you and your team during the 
guality assurance assessments of selected lma subcontractore . 



DAVID J. VENLET 

Vice Admical, U,S. Navy 




Appendix A 
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JOINT STRIKE HGHTER PROtiRAM 
200 12™ Street South, Suite 600 
Ariiugton, Virpnia 22202-5402 




Copy to: 

OUSD(ATiL) 

Director, Engineering 

JSF Weapon System Program Manager 

Director, Product ion 

Director, Program Integration 
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JPO Response to NOC Lockheed Martin (cont'd) 



FOR INSPECTOR GEHERAL CONEIDËHATIOW 

appendix ft! 

1. The F-35 Joint Program Office (JPO), in concert wtcti etie 
Defense Contract Management Agency (DCMA) and LMA, reviewed eagh 
o£ the assessmenb ' 3 findings and recominendations . I accept all 
of the findings issued relative to LMA as valid. T agree that 
all of the recommendations issued relative to lma should be 
assessed and in^Iemented to the maximum practicabie eicCenb. 
Addltionally, the F-35 JPO reviewed the findings and 
recoiwnendations issued relative to the F-35 JPO, and i also 
accept all of these findings as valid. I accept all of the 
recommendations issued relative to the F-35 JPO and will 
implement them as discyssed below. Although we may diacover, 
during our root cause analysis and öevelopment of correctlve 
action plans, that a particular aspect of a finding may he 
already addressed and fixed to satisfy requirements, I helieve 
that every finding has a deficiency that needs to be further 
addressed by our program office. Further, I intend to ensure 
that any corrections and iinprovenrents do not stay restricted to 
one löcation but are distributed and implemented throughout the 
JSF enterprise. 

2. Regarding the findings issued relative to lka, the F-35 
JPO '3 Êirat step was to establish a joint working group among 
the F-35 JPO, iiMA, and DCMA to ensure that al] of the findings 
and recommendations are understood and to start deveioping 
correotive action plans. Points of Contact were eEtablished by 
9 May 2012 for every finding within each of che three 
organizations, and discussions have already cammenced. The F-35 
JPO then notified DCMA. of our intent to accept all of the 
findings {14 May 20121 . DCMA is issuiiig Level 2 Corrective 
Action Retiuests (CARsJ for each finding [on-going, ECD 31 May 
2012). The DCMA CAR process provldes che lïovemment a 
systematic approach to ensure the Contractor takes corrective 
action in a timely iiranner. The F-35 JPO, in concert with the 
responsible DCMA organization, will implement monthly 
adjudication meetings with each location included in the 
asBessment to ensure appropriate progress is being accomplished 
in the corrective action plan for each finding. The first 
meeting with LiMA will be held in July 2012. 

3. Regarding the findings issued relative to the F-35 JFö, we 
a.re pursuing the following process. 




JOINT STRIKE FIGimïR PROGRAM 
200 12'" Street SouÜi, Suitt 600 
Arlington. Vlfginis 222(12-5402 
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JPO Response to NOC Lockheed Martin (cont'd) 



a. Of the seven findings issued xelaLive to the F-35 JPO, 
two addxess concerns regarding documentation of meetings for the 
Program Management Advieory Board and Test Readiness Reviews. 

In accordance with your recommendations , this will be addtesaed 
by 31 August 2Q12 , 

b. Four findings deal with the program iitiplementation of 
Critical Safety Item tCSI) management. Your reconmendfltions 
direct the F-35 JPO to implement a fully realized CSI plan, 
which we Euily support. An P-35 CSI-Specific Implementation 
Policy has been developed. That document ie being updated to 
conimunicate the additional CSI retïuiiemenLs aiid guidarice in 
order to addreas the DoDIG's Eindings. The updated policy will 
form the basis of the JPO plans to put the Prime Contractor on 
contract £or Phase I (Prime Contractor CSil and ?hase II 
(Supplier CSU CSI Non-recurring engineering {NREïeffort, as 
well as the CSI recurring efforts. We plan co work with our 
Prime Contractor in the next several weeks to develop a CSI 
roadmap/ Schedule for the CSI program. The F-35 JPO is working to 
update and authorize the Phase i csi HRB reguest for proposal. 

c. The P-35 JPO accepta the final finding ,4LM005, which 
identifies the following issues: "a) JPO doea not have a 
[Diminishing Manuf acturing .^ources and Mat.erial .Shortages] dmsms 
Program Management Plan as required by SECNAVINST 5000. 2£ (and 
previous revisions) or DoD 41 40. IR. b) DMSH3 management 
activities have not been funded in the low Rate initial 
Production (lrip) 4 and 5 contracts. ThereJIore, a proactive 
DMSCIS management proces s is not being implemented by LM or its 
suppliers on the F*35.* We fully accept the first part of the 
flnding regarding the lack. of a DMSMS Program Management Plan, 
but it is important to point out that we do have some 0MSMS 
activities on contract. Prior to 8 July 2011, DMS non- 
re cur ring /redes ign and management activities were performed 
under the 3DD contract ^ Af ter Chat date, DMS management 
activities are being performed under the sustainment portion of 
the LRIP 5 contract. Beginning with the def initization of the 
LRIP 5 production Undef Initized Contract Rction, which is 
currently being negotiated, all future DMS management and non- 
recurring/ redes ign activities will be included as part of the 
production contract. The JPO fully supports your recommendation 
to inïjlemeiit a robugt DMSMS program and flow that down to IMA. 




.TOtlVT STRIKK FIGIITER PROGRAM 
2<Hi 12™ Stiegt South, Suilt tiOO 
Adtnglon, Vlrginla 22202-5402 
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NOC Northrop Grumman 



INSPECTOR GENERAL 

DEPARTMENT OF DEFENSE 
iSW MARK CEMTER CRIUE 
ALtXANUHIA, VIKQINIA ZZ'iSa l&üa 

M 1 3 2Q1Z 




MEMORANDUM FOR PROGRAM EXECUTIVE OFFICE JOINT STRIKE FIGIITER 

SUBJECT: Notice of Coiicem-Qunlity Asswaiicc Asscssmclit of the F-35 Program 
(!>ruii;el No. D2012-l>'rOTAD-0003) 

Wc are issuÏDH Ihis Nolicc ol" Concern lo infomi yoii thal Ihc Dcparlmcnt of Dcfcnac, 
JnspecUir Ueiieral (DoD [G) team icleiititled issues (hal leqiiiie yonr uttentioii. Duriilg tlie 
Quatky Assurancc Asscssmcnt' oflhc F-HS program at Northmp Gmmman Coiporalion (NOC), 
El Segmido and Palmdale, Calitbmia, trom May 7, 2012 to May 18, 2012, Ehe teara identitïed 
6f) imdingü, 

Wü arranfiEd the lïndingH inlo (ïvc broad catcgories: Dncumcnt Contrnl, Requinïmenta 
Flow-Down, Process PToofing, Process Discipline, and Tool and Equipmeui ConiroL Our 
asscssmcnt of eacb of these areas is as folio ws: 

• Docwuent Conlml-Plans, procedures, and records werc not always maintaiacd 

• Req uireineuts Fl o w-D o wn-C oati'actu al and/or Federal Act) uisitLon Regula L ioü 
requireinenis were nol nlways aïrreuily llowed down niid/or implemented 

• Procébts Prooiïng- Pruducliun processes and work instruclions were not well-defmed 
and/oc inKomjiIcte 

• Process Discipline-Failure to follow eslnblished ptocedurea was prevalent throuj^out 
most qualily management proccsses 

• Tool and Equipmcm Control Tools and/or equipnient were not properly ideutified, 
pr«>ofed, and tonlrollcd 

Tbe DoD IG tsain's overall couciusioii is tliat NGC's quality niaiiagüiiieiit system is tiot 
always implemented or sufficiently defmed; hnwever, management is actively cngagcd in the 
day-lo-day piodiiclion activiüesand pruacllvt;!)' wyiking issues ai Ihuy yrise. I'heir general alliludc 
was lliai UiK issues noled by llie DoD IG aasesjïtnent team were opportunities lo improve their 
prodact. 



We condutled ihe usscssmeni in acdurdancc with the Conncil of the Inspcctor Qcncral on 
Intcgrily and [ifïiciency (Juality Slandards tor tnapection and livafuation and guidance in 
AS9101, Quality Management Systems Audit Requirements for Aviation, Space, and Defense 
Orgiini/alintis, 
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NOC Northrop Grumman (cont'd) 



Oiir findings ate atiached to this memorandum for your review and cominent, 
DoD Directive 7650.3 requires that recommendütions be resolved promplly, Plcuau pi-ovidc 
comments (hat state whether you ajjree oi' disagrec with Ihi; frndirigs and rccommendatinns. If 
yov agree with oiir recoraraendatioiis, dcscdbc what actions you have laken or plan lo lake lu 
accomplislï Ihc rccommendatinns and incliide the complelinn dates of ymii' actions. K' yoii 
disagrec with the rccomraendations or any pmi of ihem, pkase give specific rcasons why you 
disagree and propose altemative aciion ifappropriaie. Once we complete our five assassments, 
we wil I piovide a report oii our findings, aiong with any addilioiial matters that tnight come to 
our attcntioii, 

We should receive your comments hy July 1 201 2. Wc normally includc copies of the 
comments in the report. Ifyou considcr any matters to bc cxempt from public release, you 
should mark thcm clcarly for Inspector General considcration. 

Ifpossible, send a ponable document format ( pdf) file containing your comments to 
Copicü of your i;oniiiii;n(s musl have the aetual signaLurc of Lhe 
aulhorizing official for your organization. We are uiiable to accept the /Signed/ symbol in plaee 
oftlre aetual sigiiature. Ifyou arrange lo send elassified eomrnenls oleetronically, you must send 
tliem over the SECREÏ Interact Protocol Router Network (SIPRN ET). 



We appreciale lhe courtcsies extcnded to the staff. Hlea.'ie direct qucstions tn 




Randolph R, Stone 
Depiity Inspector Géneral 
Policy and Oversight 



Attachments: 
As atatcd 
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JPO Response to NOC Northrop Grumman 




JOINT STRIKE FICHTER PROGRAM 

zm n™ SUvet South. Suite 600 
Arllngtnn, Vireinfa 22202-5402 




Fr om: 
To; 



Program Executive OfEicer, F- 3 5 Lightning IT 
Deputy Inspector General, Policy and Oversight 
Department of Defense Inspector General 



Sub j : RESPONSE , NOTICE OF CONCERN - QUALITY ASSURANCE 
ASSESSMENT OF THE F-35 PROGRAM 

1. I received and smalyzed your 13 Jun 2012 Notice o£ Concern- 
Quality Assvjrance Assesstneat of the F-35 Program {Project No. 
D2012-DT0TAD-DOD3 ) regarding your visit to Northrop Grumman 
Corporation (NGC) , Ël Segwdo and Palradale, Califoraia, 7-ie May 
2012, I appreciate your team's efforts to assess conformity to 
F-35 quality management systems, contractual qyalicy clauses, 
and internal processes and procedures, to include Aviation 

Cri tical Safety Items. I view your assessmert as a valuable 
tooi to independently evaluate the areas for improvertient within 
the F-35 Program, with the ultiraate goal of producing a guality, 
timely, and cost-ef f ectlve weapon syscem for the United States 
and our allies. 

2. The F-35 Joint Proctram Office (JPOl , in concert with che 
Defense Contract Management Agency (DCMA) , NGC, and the prime 
contractor, Lockheed Martin Aeronautics (LMA) , revlewed each of 
the assessment's Eindings and recoramendations . I accept all of 
the fiiidings iaaued telative to NGC and LMA as valid. I agree 
that all of the recommendations issued relative to NGC and LMA 
should be acscssed and implemented to the maximum practioable 
extent. A detailed discussion of our path forward is contained 
at AppendiK A and is raarked for Inspector General Consideration. 

3. We win continue to work with you and your team during the 
qijality assurance assessments of the reniaining aelected IMA 
subcontractors . 




vice Admirali U.S. Navy 



J^pendix A 



Copy to; 
OUSDtATSL) 
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JOINT STRIKE FIGHTER PROGRAM 
200 12™ Street South, Suite 600 
Ariington, Virgjnla 2220Z-5402 




Director, Engineering 

JSF weapon System Program Manager 

Director , Production 

Director, Program integration 
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JPO Response to NOC Northrop Grumman (cont^d) 



FOH INSPBCTOR GENERAL CXJNSIDERATION 

Appendix A: 

1. The F-35 Joinc Program Office (JPO), in concert witH the 
Defense Contract Management Agency [DCMA) , NGC, and the prime 
contractor, Lockheed Martin Aeronautics (LMA) , reviewed each of 
the asaessmenfs Êindings and reconraendations . I accept all of 
the Eindings issued relativa to NGC and LMA as valid. I agree 
that all of the recommendatione issued relativa to NGC and LMA 
should be assessed and implemeuted to the maximum pcacticable 
extent . Althuugh we may discover, during our rooc caus« 
analysis and developraent of correct ive action plans, that a 
particular aspect of a finding may be already addiessed and 
fixed to satiaEy requirements, I believe that every finding has 
a deficiency that needs to be further addressed by our program 
office. Further, I intend to enaure that any corrections and 
iinprovements do not stay restricted to one location but are 
distributed and impleitiented throughout the JSF enterpriae. 

2. Regarding the fihdinge issued relativ© to NGC, the F-35 JPO 
established a joint working group afliong the F-35 JPO, NGC, LMA, 
and DCMA to ensure that all of the Eindings and recoinmendations 
are unöerstood and to start developing corrective action plaas. 
Points of Contact within each of the four goveminent 
organizations will be established for every finding by 18 July, 
but discusslons have already commenced. The F-35 JPO has begun 
working with DCMA at both NGC sites to understand and properly 
characterize each finding with an eye towards drafting 
Corrective Action Requests (CAHs) for appropriate findings (on- 
going, ECD 18 July 2012) . The DCMA CAR process provides the 
Govemtnent a syst erna tic approach to ensure that corrective 
actions are taken in a timely manner. The F-35 JPO, in concert 
with the respons ible DCMA organization and LMA, will implement 
monthly adjudication meetings with each iocation included ia the 
assesament to ensure appropriate progrese is being accoiiipl ished 
in che corrective action plan for each finding. The tirst 
meeting with NGC, LMA, and DCMA will be held in August 2012. 

3. Findings issued relativc to LMA have been rolled into the 
tracking process initiated during the initial visit to LMA and 
wiil be included in the LMA monthly tracking meetings. 




JOCMT strike: nCHTER PROGRAM 
200 12™ Streel .South, Suite 6I)U 
Ariington. \'Irginia 222Ü2-5402 
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INSPECTOR GENERAL 

DEPARTMENT OF DEFeNSE 
4800 M/\RK CENTER DRIVE 
ALËXANDRIA, VIRGINIA 2235Ö-1500 



MEMORANDUM FOR PROGRAM EXECUTIVE OFFICE JOINT STRIKE PIGHTBR 

SUBJ12CT; Nolicu of Com;cm-QLmli(y Assurancc Asscssmcnt ol thc F-35 Prugnira 
(f'mjcct No. iJ2(ii2-i) ro TAn-ddin) 

We are issuiiig this Notice of rniiceni to intrani yoii iliat the nepartment of' DeferiKe, 
Inspcclor üeneiai (DoD IG) twiu idenlified bsucs llmt lequiic yoiii altcnlion, Uuiing the 
Qiiiilily AssuiancG Asscssment' of the F-35 program at L-3 ComTniinicnljrnfi, Alpharcttfi, 
Oeorgiii, IVotii Jiine 18, 20] 2 to Jiiiie22,2{1l2, \\m Iwmt idenlillecl 5(i (ïndings, 

"We sorted the findings into five biond cntegorios; Process Discipline, Document Coutroj, 
Reriuirementa riow-Düwn, ï'oül and riquipmenl ConlTol, and Procass Prooflng. Our assessmenl 
of twcli of these ai eas is as follows 



• Process Discipline-Failure tti follüw eslabiislied pïoceduitis wtis prevalent thioughout 

most qiiaüty uianaecnient pioccsscs reviewed 

• Dut-Lilin;iil Coulwi PJaits. piotcdmcs, and rcyuids nul alwHjs matulaiitul 

• Ketiuiremenis l-Iow-Dowii-Contractual und/or Federal Acquisition liegulaiion 

i\;t|uircii]cnLs wcrc ndl always fiiricclly llowed tlowD and/oj' inipIcMneiiltd 
■ "IVml and liquipment ClontmI Tim\x and/nr equipinenl wcrc imt pTiipüiiy conlTolled, luid 

thé calibraiion systeni had senous delicientics 
» Proüess Prooriii^ Piodi(t:tioii pioiiesses and woi-k iiisttuctions were nol WBll-defincd 

and/oT incomplett; 

I 

TTie. t>Dn IG teaiii's oveiall tonclusion is tliat T.-3's ([iialily iTiana[;enien( Rysfem is nnt 
always implu'mciilciJ wsuffidynlly defincd, Appioxiirmldy hwlf (43 ptrwiil) «rilie finditiKs 
wefe iti pfocciss disciplLie, iiidicatiiig a laclc of attciitioii to detail, In iiddition. the calibration 
sysltiui hud seriüuü delkieiicies lliiit wucratil n foinplelt; i-evïew, «tid lïfsl ailicle iiispeclious weii; 
not hcing pcifnrmed as required. 

Oiir lindings mt attaehed to this memürandiim thr ymr review and curimeni, 
DoD Diitctivc 7650 J iMjuires ihat rccoinnigtidatioiis bs tcsolved proiuptly. PIcnsc provide 
üoniments that state wliclhei' you agrce or disagfcÉ witli the findings and i'econiniendations. If 
yuu agiee wilh oiir recomnieiidatïontj, üeseribe wlial atlinris yuii have taken ur plan to tuke to 
accomplish the fecoimnendiitions and include the complction dates of yam actions. 11 you 
disEit^ee wilh the reeommcndatious or any part ofttiem, please give specific leasions why yoü 

' Wf tüiidutlcd lla; asstssment iii aceordaiici; wilh Ihe Ciiuneil of the Itisptcloj' Cciici-al oii 
Integiity aüd Efficiency Qiiahty Htaudards Ibr Inspecttou and Evahialion and guidaiice in 
AS9iOI, Quulily JVtanugcment SyaleiTis Audil Retitiiremeflls fuT Avialiun, Spaue, and Defense 
Organizations. 
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disagree and propose altemative action if appropriate. Once we complete oiir assessmenls, we 
atteiilion. 

We should'rcccivc your comments by August 1 3, 2012. Wc noi tnally includc copics of 
the comriienis in tiie reporl. If you consider imy maners to be exempt from public release, you 
should mai"k them clearly Ibr Inspecfor (.ieneral consideraticin. 

Ifpossible, send a portable dotmnictit foriiiat (.p'if) fllc conlaiuing your comments lo 
Copics of your comments muüt have the actual signutuic of the 
aulhoi'izing oiïlcial lor your organii'atiün. We are unable lo accept the /Signed/ symbol in place 
of tin; actual signaturc. If you arrangt; lo sctid classificd comments clcctroiiically, you must sutid 
them over the SECRET Internet Protocol Router Network (SIPRNET). 

We appreciate the couitesics extcnded to the staff. Please direct questsons to | 



^ Randolph R. Slone 
^Depuiy Inspccior General 
Pulicy and Ovcrsighl 

Altaelimenl: 
As stated 
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JOINT STRIKE FIGHl^R PROGRAM 
ZOO n™ Streel South, Suite 600 
Arlingtan, Virfiinia 22202-S4l)>2 




Prom 
TO: 



Program Executive Officer, F-35 Llghtnlng II 
Deputy inspector General, Policy and Oversight 
Depar tuten t □£ Defense Inspector General 



Sub 3! RESPONSE, NOTICE OF CONCERN - QüALITY ASSUEAHCE 
ASSESGMENT OF THE F-3?i PROGRAM 

1. 1 received and analyzed your 11 Jul 2012 Notice of Concern- 
Quality Assurance Assessmenc of the F-35 Program (Project No. 
D2012-DT0TAD-0003) regarding your visit to L,-3 Cormnunications , 
Alpharetta, Georgia, 18-22 June 2012. I appreciate your teaiii's 
efforts to assess conformity to F-35 cruality management systems, 
concractual duality clauses, and internal processes and 
procedures. I view your asseasinent as a valuable tooi to 
independent ly evaluate the areas for improvcment within the F-35 
Program, with the ultimate goal of prodycing a quality, timely, 
and co3t-ef Éective weapon system for the ünlted States and our 
allies. 

2. The P-as Joint Program Office (JPO), in concert with the 
Defense Contract Management Agency (DCMA) , L-3, and the prime 
contractor, Lockheed Martin Aëronaut ics (LMA) , reviewed each of 
the assesstnent's findings and recofflinendations . I accept all of 
the findinga issued relative to L-3 and LMA as valïd. I agree 
that all of the recommendatians isstied relative to L-3 and LMA 
should be assessed and itnplemented to the majcimum practicable 
ejttent. The JPO acknowledges the findings issued against DCMA 
Fort worth and dCma Atlanta, and will collaborate with dCma to 
ensure recommendations are assessed and implemented to the 
ttiaximurti practicable extent. A detailed discussion of our path 
forward is contained at Appendix A and Is inarked for Inspector 
General Consideration. 

3. We will continue to work with you and your team during the 
[juality assurance assessments of the rejnaining selected LMA 
subcontractors . 



DAVID J. i/eNLET 
Vlce Admiral, U.S. Navy 
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JOINT STRIKE FIGHTER PROGRAM 
200 12™ Street South, Sutte 600 
Artington, Virginia 22202-5402 




Appendix A 

Copy to: 
OUSD(ATiL) 

Di rector, Engineering 

JSF weapon System Program Manager 

Director, Production 

Director, Program Integration 
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FOR liflSPECTOR GENERAL COMSIDERATION 
Appendix A; 

1. The F-35 Joint Program Office (JPO), in concert with the 
Defense Contract Management Agency (DCMa) , L-3, and the prime 
contracLor, Lockheed Maitiri Aeronautics [LMAf , reviewed each of 
ehe assessment ' s findings and recniranendations , T accept 8 11 of 
the findings issued relacive to 1.-3 and LMA as valid. I agree 
that a.11 of the recononendationa issued relativa to L-3 and LMA 
should bm assessed and implemented to the maxiraum practicable 
extent. Although we may discover, during our root cause 
ajiölysis and development of corrective action plans, that a 
particular aspect of a finding may be already addressed and 
fixed to satisfy regiui renen cs, I faelieve that every finding has 
a deficiency that needs to be further addressed by our program 
office. Further, T ineend to enaure that any corroctions and 
improvements do not stay restricted to one locacion but are 
distributed and implemented tTiroughout the JSP enterprise. 

2. Regacding the findings issued relative to L-3, the P-35 JPO 
established a -joint working group among the F-35 JPO, L-3, LMA, 
and DCKA to ensure that all of the findings and recoiratiendationa 
are understood and to start developing corrective action plans. 
Points of Contact within each of the four organizations will be 
established for every finding by 10 August 2012, but discussions 
have already commenced, The P-15 JPO has begun working with 
DCMA at L-3 to understand and properly characterize each tinding 
with an eye towards drafting Corrective Action Requests (CAEs) 
for appropriate findings (on-going, ECD 31 August 2012) . The 
UCMA CAR procesE provides the Government a systematic approach 
to ensure that corrective actions are taken in a timely jnanner. 
The F-35 JPO, in concert with the responsible DCMA organizatiun 
and LMA, will implement monthly adjudication meetings with each 
location included in the assessment co ensure appropriate 
progress is being accomplished in the corrective action plan for 
each finding. The first meeting with L-3, LMA, and DCMA will Tae 
ïietd in .September 2013, 

3. Findings issued relative to LMA have been rolled into the 
traclcing process initiated during the initial visit to LMA and 
will be included in the LMA monthly tracking meetings. 

4. Findings issued relative to DCMA Fort WorLh and DCMA Atlanta 
are being actively addressed. DCMA Atlanta has ensured proper 




JOINT STRIKE FICHTER PKOCRAM 

2O0 13™ Sti-m Stiulli, Suiiï 600 
Ariingtan, Vir^nla 22202-$4U2 
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JOINT STRIKE FICHTER PROGRAM 

200 12'^" Street South, Suite 60D 
Ailineton, Vii^nia 22202-5402 

procedures are in place and personnel properly trained co 
conduct the required inspeccions. They have developed a revisetï 
(Juality Assurarice Surveillance plan for L-3 Display Systems and 
are developing an internal procedure for "Delegation Review and 
Receipt," which will be completed by 3i August 2012. 



DMCA Fort Worth has reviewed thelr Letters of Delegation to 
ensure a11 inCorination is correcLly reflectect, which is expected 
to be complete by 31 August 2012. DCMA Port Worth is also 
modifying their P-35 Suppliers Information Delegation to include 
a Monthly Report ing Reguirements column, and wiil follow up 
within onm business day if tfie deadline for r sporting is missed. 
Finally, DCMA is implementing a new procedure to extract/corapile 
and forward to the cognizant DCMA office performance data 
relating to those assigned suppliers whose performance is below 
LM's Supplier Quality Rating or that have received related 
product def iciancy reports . They are updating their 
Surveillance strategy and desk top instructions to reflect thls 
change, which will be coinpleted by 30 September 2012. 
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INSPECTOR GENERAL 

DEPARTMENT OF DEFENSE 
4S0O MARK CENTER DRIVE 
ALËXANDRIA, VIFiGINIA 223Sa-ISMi 



JUL21i2Q12 



MEMORANDUM FOR PROORAM EXBCUTIVE OFFICE JOINT STRIKE FIGIITER 

SUBJECT: Noticc of Coiiccm-Quality Assurance Asscasmcnt of Ihc F-35 Pro[5:!iin 

(i'mjeci Nii. n2(n7.-iyn)'i'Ai)-n:i(n) 

We ace issniug this Notice of Coneern to inform you rhst tlie Depfii rment nt" Defeiise, 
Instieclor Oencral (DoD IG) leaiu idcnttllcd an issue ihal itquiies avtcntion. Diiiing h pre- 
asstssmcnl visit at HnncjTVclI, F'liocnix, AA, oti .luly 19, 2012, we Foiind Ihat DCMA Phnciiix 
hiw iiui been (.klegatetl uveisigiil uf produeliou üüui DCMA, Lockheed Mai liii Furt Wiirlh. Tliis 
lack ol'clelegation criukl resiill in qirality escapes ;is (>(!MA surveillance and qualily ovcrsighl 
fiiDcf ioiis ftic uo: bciiig ptrfonncd, Jt sJioiild bc iwtctl tlml DCMA Phociiix does liave a 
delegatioii tbr fepair mid rework. 

Ow fiüdiög is attciched to this luemoraiidLini for yoiir re view nnd comment. 
DoD DircclivL' 7650.3 rcquiivs (hal iccommendiiliunü be resulved promplly. Plense priivide 
commcnts ihat stalc whclher you agi'uc ur disagrec wiih Ihc niidinfiï and ïcconinieiidation. IT 
you agret vvilli our iccoiuiiuïndalioii, dwcrlbc whot nctioiis you litivc talccii or ploti to tcikc to 
BEXumpbsh Ihe reeommendatior and int;hide the cumplelion dii[es Dl'youT actiuna. 11 you 
disagjec vvitli the icgununcjidalioii or aiiy pait of il, picase gjve spccific rcasoiis wliy you 
dtragree find propose aitcrnative aciion if appropriatc. 

We shotild reecive yourcomments by August 13, 201 2. Tf you cnnsiderany matters 
exempl Jrom publii; release, you skould mark theni clearly forinspector Ceneiul eomideratioa 

If jioKsiblc, send a portable document format (.pdf) file eontaininR ymir ctiiriTicnls to 
Cupie;: uTyoui' ctimnients mu.sl httve Ihe actual st^naUtrc of Ihc 
authuri/.inj:, oiTicial liii' yuiir urganizalioii. We are iiimble lo aeeept ihe /Signed/ sytübol 'm place 
ot'aaactual siguoturc, If yon giraLHje to seiid classüjed cormueuts eleetrouicjUly, you musl sand 
thein {n'crihe SRCRFT Intemel l'mtnool Rnuter NetWork (KIPRNR'O. 

We appreciate iKe cinii (esies exteiided [« ihe st&iT, Please di|-ect questioiu lo^^^^^^ 



jii llandoli^ft R. Stonc 
Depuly Inspeclur Ocnertil 
1'üliey and üversighl 

Attachment: 
As slated 
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F-35 JOtW PROGRAM OFFICE 
iW 12"' Sinxl Soulh, Suilt 60» 
AribiKtoD, Virgina 22202^)2 




Frotfi: Program Executive Officer, Lightnlng ir 

To: Deputy Inspector General, Policy and Oversight 
Oeparcment of Defense Inspector General 

3llb j : RESPONSE , NOTICE OF CONCERN - QUALITY ASSÜEANCE 
ASSESSIIENT OF THE F-3S PROGRAM 



1 . I receiveö and analysed your 26 Jul 2012 Notice of Concern- 
Quality Assurance Asseasment of the F-35 Program (Project Ko. 
D2D12-iyrUTAD-0003) regarding your visit to Honeyweli, Phoenix, 
AZ, on 19 July 2012. I appreciate your teara's efforts to aesess 
conformity to F-35 ciuality management systems. I view your 
assessment as a valuaiile tooi to independently evaluate the 
areas for improvemeait within the F-3 5 Program, with the ultimate 
goal of producing a quality, tirnely, and co5t-e£f ective weapon 
system for the üniCed States and our allies . 

2. The F-35 Joint Program Office (JPO), in concert with the 
Defense Contract Management Agency (DCMA), reviewed the single 
finding and recoirnnendation. The JPO acknowledges the finding 
issued against dCma Ft Worth, and will collaborate with DCMA to 
ensure recommendations are aaseased and implemented to the 
maximum practicable extent, ft detailed discussion of our path 
forward is contained at ^peiidix A and is marked for Inispector 
General Consideration. 

3. We will continue to work with you and your team during the 
guality aaaurance aasessments of the remalning selected Lockheed 
Martin AsronatitScs subcontractors. 



DAVID J/VENLET 

Vice Amniral, D.S, Wavy 

Appendix A 
Copy to; 
OUSD(AT&Li 

Di rector. Engineering 

JSF Weapon System Program Manager 

Director, Production 

Direccor, Program integration 
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FOR INSPECTOR GENERAL COMSIDERATION 
Appendix A: 

1. The F-35 Joint Program Office (JPO), in concert wir.h the 
Def ense Contract Managesnenc Agency (OCMA) , reviewed tho single 
findiïig and ree ommendat ion. The finding issued reiative to DQIA 
Ft Wornh is being actively addressed. They have added a 
supx3orting Letter of Delegation for the Turbo Machine. The 
subaeqpieïit Letter of Delegation from DCMA Santa Aiia was accepted 
by DCMA Phoenix on 09 August 2012. We now have surveillance over 
the entire PTMS to now include repair of the Turbo Machine. The 
JSF JPO will continue to track the completion of the DCMA 
Correct ive Act ion plan as part of the overall status of findlngs 
from all of the DoD iG Quality Assuratice Aasessmeiits. 




JOINT PROGRAM OFFICE 
200 12™ Streel South, Suke 60Ö 
Arliuj^, Vvfitia 22202-5402 
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INSPECTOR GENERAL 
CEPARTMËNT OF DEFENSE 

4910 mm CÊNTËfï DRWC 
ALIIXANDRIA. VIRGINK £2350-1500 



ocT 2 z m 



MEMORANDUM FOR PROORAM BXECUTtVE 01-TICK JOINT STRIKK KIOH'CKR 

S U Ü JEC Not ice O r C O! icerii-Q 11 n l i ty AsEU l aiicc Assössraent of the F-3 5 Progca m 
(Pioject No. D20 12-nT()TAD'()Ü03) 

We ni-e issiiing tliisNoticc ofCoacem to intoim yoii tliat the Departineiit ofDetfense 
liisiTeptor {'iHneral (llfill IG) team irictififiert issues Ihal retiuiii; your lidmiliijn, rjiiriiig llie 
Qiiality Asaiirance Assiissuiciit' of llic F-35 progioin ni BAE Sysleins, Snmlesbiuy, United 
Kiiigdom, fi'om September 1 S, 2012 to Septembei* 28, 2012, tlic team identificd H2 findinys. 

Wc sortccl l!iB fitlctiiigs into fivc liroad catcfioiics: Proccss Discipline, Documeiil Coiili'ol, 
Ret|uircmunls Fluwdovvii, Tuul and Ei|iiipmeiii Caiilrol and Calibralioii, aiid Piocess Pioofiiig. 
Uur assessmeiil ofeacli or these areas is as Ibtlows, 

• Proccss Discipline Pi ocesses are in place, bilt (bcus should be on atteniion to detail. 
■ DtJtiinicni Control-Plnns, procetUrrcs, aud records aie nci nlwnys maiiitained. 

O RciiiiiirinDiiiH Flüwduvvn-Conti'iiotijal retiuiieiiieiits me tiot always conectly tlowed 
down and/oi' im|ilcinctiicd. 

• Tool/rii-|iiipiHciit (.'omiwI/Calibration-Tools and/or cquipmcnt are iiot propcrly 
conlroiled, awi liie catibinliou system luid scrious dcilcicncies, 

• Prui;eas Pioofing Protssaes are leliant on operator kiiovvledge and traininjt rathcr 
thaii documeiited processes. 

Tbc DoD 10 leam's overall cioncikision ïs tbal BAK's qiiality nianaaenicnt system is i»ot 
always iniplcmentcd or siinieicntly ticlined. Tlic jModuclion facility lacks tooi controls. the 
calibrutioii system surfers from sciioiis deficiencies lliat warrant n complete rcview, niid soltwai'<f 
lesl elTorls neeti iticreased rifjoi' anci inanagemant parlicipalinn, I lowevei'. we fmiTid Ihal llie 
design i'eview proeess is sysleiiiiilie, orgiinizetl, and eolisistciilly implcuiealcd; tlic life cycic test 
elïort was assessed to be efffective; and worktórce and management dedication, knowlcdge, 
skilla, and abiliticsoie driving BAE's auocess, 

0(ir iiiidiiigs ine attnched to this raeinoraiitJuni (br your leview and commeiil. 
IJoD Direclive 7650,3 rcquiics Ihal rccoimncndalioiis hc rcsnlvcd piomplly. I'leasc provide 
coinraoiils iha: sliite whcthcr yoii nyee or disagiee witli llie ttiidines and recoinmendaiioMs, If 
you ngree wilh oitr itconnneiKtatinns, desciibe wliat aciinns ym have taken or |ilan lo lakc to 
aceoinplish the rccortimeiidaiions and include tlie compleiion dates of your actions. 11' ym 
disaKlïe wifli Ihc reoominendatiors or any part of Ihem, please give spettfie reasons wliy yim 

' Wc eondnelcd the asscssmetit in accoi'dancc wilh the Cooncil of the Inspcclor Ociieial on 
Integi'ity and Efficiency Qunlity Standaixis for Inspection and Fvalunticin and guid^nec in 
AS9ID1, Qiiality Mnnagemeiil Systems Andil Retjuii'cmoiits for Avtalion, Spacc, and Dcfeiisc 
Orgatii^attons. 
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diSHgicc and praposc allcmnlivi! nctioii if apprapriale. Uiicc; we c{ini|)lete uur asscKsiiienls, wc 
will provitlé it itfpcjil on ovu- findiiiBs, altnig vvilh Jioy addiliuiinl üiiitlt^is llinl iiiigUt cCiiilê to Cilli' 
ulliinlion. 

We should reccive yoiif comments hy Navembcf 21 , 201 2. We normally include eopiea 
af tlie comiiienis iii ihe vepoji, It" yoii consiclet miy mmteiï; lo be exetnpl ftom public release, yoti 
^tiuld nvAsk (hem i^learly fcii' Itispeclov ( leiiei'al cnii^ideiatinn. 

If posslblc, send a poriabie documeiu foi'iuai (,pdf) fïlc comairiiiig your coinincius lo 
Copics ol'yoiit' cojDinciUs must linvc th: acliin! sij^natuiv of ttic 
Hiillioriziny. nlTicinl liii youi ni'jjani/alion, We ure im:ihle In accejil llie /Sigiied/ syiidml in place 
of Uit; uuliml sigiiiiUin:. If yuii tiiiuii!>c k> üciid ;:lii»»iric(.l ci>]iiiiii;iilï! eluctiunically, you iuusl sejld 
llieni ovei-llic SCCRET Imcjucm Prolocol Roiitcr NcUvork (SIPRjNET). 

Wc iipjJi ucialc (hf coiii ltsics L'xtciidcti lu llic sliilï. Pkiisc tiiri;t;( qticsliolis lu | 




Riind(it|ih R. Stuiii; 
Dcpiily Inspccliir CIcnorid 
Policy fliid Ovcrsight 



AliHchtnenl: 
As stated 



98 I DODIG-2013-140 



Appendixes 



JPO Response to NOC BAE Systems 




JOINT S TRIKIC FIGHTER PROGRAM 
2W 12^" Strwt South. Suite 600 
Arlingiton, Virginia 32202-5402 




From 
To : 



Program Executive OEficer, F-35 Lightning II 
Deputy Inspector General, Policy and Oversight 
Department of Defense Inspector General 



Subj : RESPONSE, NOTICE OF CONCERN - QÖALITY ASSORANCE 
ASSESSMENT OF THE F-35 PROGRAM 

1 . I received and analyzed your 22 Oct 2012 Notice of Concern- 
Quality Assurance Assessment of the F-35 Program (Project No, 
D2012-DTOTAD-0003) regarding your visit to BAE Systems, 
Samlesbury, United Kingdom, 18-28 September 2012. I appreciate 
your team's efforts to assess conformity to F-35 guality 
management systems, contractual quality clauses, and internal 
processes and procedures, i view your assessment as a valuable 
tooi to independen tly evaluate the areas for iraprovement within 
the F-35 Program, with the ultimate goal of producing a quality, 
tiraely, and cost-ef Eective weapon system for the United States 
and our allies. 

2. The F-35 Joint Program Office (JPO), in concert with the 
Defense Contract Management Agency (DCMA) , the United Kingdom 
Ministry of Def ence (MOD) , BAE, and the prime contractor, 
Lockheed Martin Aëronaut ics (LMA) , reviewed each of the 
assessment 's findings and recoitimendations . I accept all of the 
findings issued relative to BAE and LMA as valid. I agree that 
all of the recommendations issued relative to BAE and LMA should 
be assessed and implemented to the maximum practicable extent. 
The JPO acknowledges the Einding issued against DCMA Northern 
Europe, and will collaborate with DCMA to ensure recommendations 
are assessed and implemented to the maxiinmn practicable extent. 

I accept the finding and recoiwnendation relative to the Joint 
Program Office as valid. A detailed discussion of our path 
forward is contained at Appendix A and is rrtarked Eor Inspector 
General Consideration, 

3. We will continue to work with you and your team during the 
quality assurance assessments of the remaining selected F-35 
suppliers . 



DAVID J. VÈNLET 
Vice Admiral, ü.S. Navy 
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JOINT STRIKE nCHTER PROGRMl 
200 12"' Street South, Suite 600 
Arlingloii, Virgin!» 2220^540Z 




Appendix A 



Copy tor 
OUSD(AT&L) 

Di rector, Engineering 

JSF Weapon System Program Manager 

Di rector, Production 

Di ree tor, Program Integration 
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JOINT .STRIKE FICHTER PROGRAM 
lOfl 12"' Street South, Siiitc 600 
Arliiigton, Virginia 22202-5402 




FOR INSPECTOR GENERAL CONSIDERATION 
Appendix A: 

1. The F-35 Joint Program office (JPO), in concert with the 
Defense Contract Management: Agency (DCMA) , the United Kingdom 
Ministry of Defence (MOD), BAE, and the prime contractor, 
Lockheed Martin Aeronautics (LMA) , reviewed each o£ the 
asBesstnent ' s findings and recommendations . I accept all of the 
findings issued reiative to BAE and LMA as valid. I agree that 
all of the recommendations issued reiative to BAE and LMA should 
be assessed and impleraented to the maximum practicable extent. 

I believe that every finding has a deficiency that needs to be 
further addressed by our program office. However, we may 
discover during our root cause analysis and development of 
corrective action plans that a particular aspect of a finding 
«lay already be satisf actorily addressed and fixed. Further, r 
Intend to ensure that any corrections and improvements do not 
stay restricted to one location but are distributed and 
implemented throughout the JSF enterprise. 

2. Regarding the findings issued reiative to BAE, the F-35 JPO 
has established a recurring joint working group among the F-35 
JPO, BAE, LHA, MOD and DCMA to ensure that all of the findings 
and recommendations are understood and to develop corrective 
action plans. Points of Contact within each of the four 
organizations were established for every finding on 5 November 
2012, and discussions have commenced. The F-3 5 JPO has worked 
with DCMA at BAE to understand and proper ly characterize each 
finding and DCMA has issued Corrective Action Requests (CARs) 
for appropriate findings , The DCMA CAR process provides the 
Government a systeraatic approach to ensure that corrective 
actions are taken in a timely manner. The F-35 JPO, in concert 
with the responsible DCMA organization, MOD, and LMA, will 
itnpleraent monthly adjudication meetings with BAE to ensure 
appropriate progress is being accomplished in the corrective 
action plan for each finding, The first meeting with BAE, LMA, 
MOD, and DCMA will be held in December 2012. 

3. Findings issued reiative to LMA have been rolled into the 
tracking process initiated during the initial visit to LMA and 
will be included in the LMA monthly tracking meetings. 
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JOINT STRIKE FIGHTER PROGRAM 
2(W 12^" Strwt Soiith, Süitc 600 
Arlington, Virgiiiia 22202-5402 




4. The Einding relative to DCMA Norbhern Europe (E)CMA NE) is 
being actively addresseö. DCMA NE has been granted the authority 
to increase stafE by 2 Engineers and 2 QA representatives in an 
effort to increase the number of NonconEormance Katerial (NCM) 
actions reviewed. As well, dCma ne will submit an altemate 
approach plan, per DCMA polioy, to DCMA HQ Operations 
Directorate for approval Eor the reason that 100% review of the 
supplier's NCM submittals is not Eeasible due to increases in 
WCMs related to increases in aircraEt production. This action 
will be completed by March 15, 2013. Regarding the issue of 
Eailure to delegate Material Review Board (MRB) authority to 
DCMA Representatives at BAE System (BAES) suppliers; dcma ne is 
working with DCMA Ft Worth to iflake certain the proper contract 
raquirement is levied on BAES by Lockheed Martin (LM) , The 
requirement levied on BAES will stipulate that all KCMs shall be 
submitted to the onsite Government representative for 
concurrence and will be obligatory throughout the BAES supply 
ctiain. This will allow DCMA NE to issue delegations, reinforced 
by applicable contract documents, which will enable the proper 
level oE oversight of Material Review Board processea. These 
actions will be completed by 15 March 2013 . 

5. As the F-35 Program defines its Critical Safety Itetn (CSIJ 
development and implementation reguirements, the JPO will 
continue to take DoDIG's recommendations into consideration . 
The JPO plans to collaborate with DCMA Headquarters and dcma Ft 
Worth to determlne how best to leverage the Engineering Support 
Activity's CSI delegation (to LMA and dcma) on minor 

nonconf orwance and class II engineering change proposals within 
the CSI policies to ensure that adequate checks and balances are 
in place to support the F-35 CSI program. The CSI 
respons i bi lity delegation will be formally flowed down to the 
Prime Con tractor and DCMA through F-35 contraots and other 
program plans related to CSI and the JPO/DCMA Memorandum of 
Agreement (MOA) . 
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INSPECTOR GENERAL 

UITPAK rMË NT OF UErENUE 
18M KltARK GEN f ÉR URIVÉ 
Al EXANDRIA, UIRQINW ZZ3fiD ISM 

NOV 2 D 2012 




MEMORANDUM FOR PROGRAM EXECUTIVE OFFICE JOINT STRIKE FIGHTER 

SUiïJKCT; Nolice ol" Concern Qiiaüly Assumiice Assesameiit oflhe F-35 l'rogiam 
(Hmjecl No. U2O12-DT[)TAU-0tl03) 

Wu are iüsLiing lliiü Nu[icu urCunmn lu iiirurni yuii lltul ihu Dt^pmUiictil orDeR-iise 
Inspcctoi' General (DoD IC) tenni identilled issues tlvat rcquire yoxir atteminii. DLcing the 
Ouality Assiiriince Assessmcnt' ot'lhc P-3S prognit)! nl Honeywell AcrnHpHce, Yeovil, IJniied 
Kiiigdom, IVotn Ocinbor 7M, 3012 lo November 2, 201 2. the Kmw idcntificd 38 flndiiigs. 

We sorled llie (ïmiiiigs into five broad categoiies: IVocess IJiscipliiie, Docitment Control, 
Rctjiiirenicnts Flowtiowi, Tnol and E(]iiipnieiit Control and Calibrfllïoii, and Proecss PniofliiH. 
üur asscssnieiit ot cach of these nrcas is as fbllows, 

• l'roccas Discipliiic-Failuic lo follow cstablishcd jiroccdurcs was prevaleiit lliiovighoiK 
niosi qii;dity niaiiagenii:)il piocesses reviewed. 

• Document Conirol 1'lans, iirnceduies, and records are nol always maiiilained. 

• lietiiiiremenis Plowdown-Conlracllial réqtiirflUlenlS are nol always correctly flowed 
t!o wn a iid/or i mpleiiiei lied . 

• Tuol/Eq ui pment Cüulrol/Cal i bm t ioii-Tüol a and/or eq u ipn len t a i e not properly 
conl rolled. 

• Process Proodng- Prodtiction proccsscs aild wotk iiisinictioiis wcrc tiot wcU-dcfincd 
and/or incom|)lctc. 

'nic DoD 10 icam's oveiail conclusioti is (liat Honey>veirs qualiiy managemenc systeui is 
nol always impleniciited or siiftlciciilly dcfined. Over half (60 percent) of the riiidings were in 
proeess discipline, indicating a luck ufallenlion lo detail. In addiiion, llu' design review process 
and irniiiiiig syalciit hi^d scrious deficiuicics liial warrant a cuiiiplete ^c^'ic^v. 

Oiir Inidings aiciiUached to lln.s' memuiHndiiTu loryuLir revit:w üiid euinnii:nl. 
DoD Direelix'c 7650.3 rcquiies diat rccoinineiKlations be resolved prompt I)'. P lease provide 
commcnts thul .'itnto whiillicr you agrce ordisagree with die tlnJings and rccoiunientlatiuns. If 
you agree with oiir rccoiiiincnclalioiis, dcscribc wlial actions yen bave taken or plan to lake to 
aeeuiiiplish tlie i^cumiiicndaliuns and ineliidL' the cuinplelioii üiiles of yoiir aeliutis. If yoii 
(lisagree witb the leconiinendatiojis or any part nfthein, pleascgivc specific rea,soiw why you 
itisiigrce iiiid ])iopusc Lillcinatii c action if apjiropiiali;. Onec we eoinplcle om aasessnieiils, we 



' We cüiiducted tlie iissessmeut in accoi-dance witb the Coimcil ofthc Inspetlor General on 
Intcgrity and Ffficicncy Qualiiy Standaids for Insijcclion and Evaiitotloii tind guidnnce in 
ASVIOI, Qtiality Management Systems Audit Requirenients for Aviutio», Spuce, and Defense 
OrgHtii'.uitioir!>, 
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will piovide a lepoil on oiir fiiulings, aloiig witti arsy additioiial malteix tliat miglit oome to oiir 
altenlioii. 

We shoultt leceive yoiii coiniiietits by December 2 1 , 201 2. Wc iioiinally iiichidc copics 
of llie coininetils iii the lepoi l, If you coiisider aiiy m.Utci's lo be cxciiipt firom public release, yoii 
shoiikl mark llicm tleni ly foi' hispecloi' Genei al cousideration. 

irpassibic, scnd a portable document format (.pdf) filc coiitfiinins yoiir eoinmenls lo 

('opicü ol'your coiiimcnts iinist have ihc aetiinl signtitiire uriiie 
aiitliori/.ing oHicial tbr yoiir orgaiii'/atioii. Wc are imabk to accept tlic /Kigitcd/ syinbol in place 
ol tbciactual sigiiaturc. Ifyoii arrangc to scnd elassified coniinciits clcctroiiicallyj ymi imist scnd 
tliem over the SECRïïT Internet Pititocol Roiitcr Nctwoik (Sll'RNIïl). 

Wc apprceifite tlie eoLirtcsies cxtciKietI to tlie staff. Pkiise diiect qiieslious lo| 




Randcilpb R. Stone 
Dcpiity Iiispcetor General 
Policy and OveisiRlit 



Altachmeiit: 
As slalcd 
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JOINT STHIKK FIGHTER PROGRAM 

20O 12™ Street South, Suite 600 
Ariington, Virginia 22202-5402 




From: Program Executive Officer, F-35 Lightning If 

Td: Dcputy Inspcctor General, Policy and Oversight 1 ^ '^"^ 

Department of Defense Inspector CetieraJ 

Subj: RESPONSE, NOTICE OF CONCERN - QUALITY ASSURANCE ASSESSMENT OF 
THE F-35 PROGRAM 



1. t reoeived and analy/ed yuur 20 Nuv 2012 Notice urConuem-Qualily Assurance Assessment 
of the F-35 Program (Project No. D2012-DTOTAD-0003j regarding yoiir visit to I-loneywell 
Actospacc, Ycovil, United Kingdom, 29 Octobcr - 2 November 2012. I appreciatc your teain's 
efforts to assess confortnity lo F-35 quality management systems, conitBctual quality clauscs. 
aiid intemal pruueüses und prokM^Jurts. I view your assessment as a valuable tooi (o 
independently evaluate the areas ibr improvetDent within the F-^ 5 Program, with the ultimate 
goal of produclng a quality, timcly, and cost-efiective weapon system for tlie United States and 
our at li es. 



2. The F-35 Joint Program Office (JPO), in concert with the Defcnse Contract Management 
Agenuy (DCMA), dte United Kingdom Miiiistry of Defence (MOD), ihe F-35 prime contractor, 
Lockheed Martvn, and its suppüers, Honeywell, and BAE Systems, reviewed each of ttie 
asscssmcnt's findings and recommeiidaiions. I accept all of Üie findings issued relative to 
Honeywell and BAE as valid. I agree that all of the recommendalions issued relative lo 
Honeywell and BAE should be assessed and implemenied to the maximum pracücable extent, A 
detailcd disctLssion of our path forward is cnntaincd at Appendix A and is marked for Inspcctor 
General Coiisidemtiun. 



3. We will continue to work with you and your team duriog the quality assucauce assessments of 
the remaining selectcd r-35 su^liers, 

CHRISTOPHER C. BOGDAN 
Lieutenant General, U.S, Air Force 



Appendix A 

Copy to: 

OUSD(AT&L) 

Di reetor, Engineenng 

JSF Weapon System Program Manager 

Di ree lor, Pruduution 

Director, Ptogram Inlegration 
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JOINT STRIKE FIGHTKR PROGRAM 
m 12™ Street South, Suite 600 
Ariingtoa, VirgjiÜH 222Ü2-S402 




FOR rNSPECTOR GENERAL CONSiDERATlON 



A ppendix A: 

1 - The F-35 Joint [>rogram Office (JPO), in concert with the Defcnsc Contract Mimagement 
Agency (DCMA), llie United Kiugdoiti Minist/y of Defeiice (MOD), the F-35 pnme contractor, 
Lockheed Martin Aerospace (I.MA), and its supplicrs, Honeywell and BAF Systems, reviewed 
each of the assessment's findings and recommendations. ( accept all of the findings issued 
retahvc to lioncywdl and DAE as valid. 1 agrcc that all of the recommendations issucd rclativc 
to Honeywell and BAE should be assessed and implemented to the maximum practicabte extent 
I bchcvc that cvery tinding has a dcficicncy thal necds to bu hirthtr addressed by om program 
office. Howcvcr, we may discover during our root causc analysis and development of oon-ective 
action plina that a particular aspect of a finding may slrcady be satisfactorily addressed and 
tïxed. Further, I intend to ensure that any corrections and improvcments do not stay rastricted ta 
one location bul are distribuled and implcmcnttd thruu^out the JSF crilcrprisc. 

2. Regarding the findings issued relative to Honeywell, tJie F-35 JPO has established a reountng 
joint workiitg group among the F-35 JPO, Honeywell, DAE, LMA, MOD and OCMA to ensure 
all of Ihc findings and recommendations ure undcrstood and to slait dcveloping correctivc action 
plans. Pointe of Contact within each oflhc four organizations have been established for evety 
findmg and discusüions have alrcady commeoced, The F-35 JPO has begun woriang with 
DCMA at Honeywell to underetand and properly characterize each findtng with an eye towards 
drafting Correctivc Action Requests (CARs) for appropriate findingïi (on-goiiig, ECD December 
2012). The DCMA CAR proccss provides the Government a systemalic approach to ensure that 
coirective aclions are taken in a timcly tnanncr. The F-35 JPO, ia concert witïi the rcsponsible 
DCMA organization, MOD, and Honeywell, will implemenl monthty adjudication meetings with 
each location included in the asaessment lo ensure appropriate progress is beiiig aecomplished in 
the correctiVÈ action plan for each finding. The tïrst meeting with Honeywell, BAE, MOD, and 
DCMA will be held in January 201 2. 

3. Regarding the finding issued relative to BAE Sysietns; it has been rolled into the tracking 
process initiatcd during the firstvisit to BAE Systems, Saralcsbmy, United Kingdom, and will be 
included in the BAE monthly [racking meetings. 
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4800 r*VRK nFNTER HBIVF 
ALÊKAMDRW, VIHGINIA Z23öa 15C0 



INSPECTOR GENERAL 



DEC 1 9 2012 



MRMORANDl rM FOS PROOHAM RXRCIJTIVR OFFICR JOINT STOKE PTGHTER 

SUBJECT; Nolice □rCoiKeni-Qiialily Assiuaiice Assessnienl of llic P-3S Program 
(Projecl No. D2O12-liT0TAI5-OU03) 

We are isMiiitn; ihis Nolice orConcern io inform ynii llral Ihc Dcpai'tiiienl nrDefciisc 
liispecior üeiieifll (DoD IG) leam idemificiJ issues llial rcquirc yom' altemiüii. Dwing ihe 
Qualily Assiirancc Assessmeiil' orilie T-IS program at United Technologies Corporalioii, 
Aeit)space Syslems (UTAS), peifonncd nl tlie lndt;|)endeiice, Oliio and Tojl Woilli, Texas 
facililics Proni Novoinbcr 27, 2012 to IJeceinher 5, 2013, tlic rcani ideiilificd 51 findings. 

Wc sailcd the findings itito fivc braad ca(cgoi'ics: l'iocess Discipline, Docmncnl Conlrol, 
Requircnieiits [-iowciowii, Tool and Lqtiijmicni Conlrol and Calibratioii, and Process Proofing. 
Otu' iissM^ssmËiit üfeach of these areas is as folbws. 

• Process Disciplinc-f'ai liire to follaw eslabl islic<ï procedures ivas prevalent tlimngiwtt 
most quality managcmcjil proccsscs rcvicwcd, 

■ IJöcnmcnl Conlrol Plans, procedures, and records are not always niainlaiiicd. 

■ Rcquivcnicnts Plnwdown-f^onli'aclnal rcqiiiicmcnls are nnl always cnrrectly llowed 
down aiid/Of implenientcd. 

• Tool/Rquipmenl Coiitrol/Cnlibmiion Tools and/or equipniem are nol proporly 
conlrol led. 

• I'roctss Proofmg- Production proeesses and work instrnctions were nol well-dcfiiied 



Tlie Dol) Ki Icani's overall concluRÏtin is that tJTAS's qualily managciiienl syslem 
requircs significant focus in a coiiple of key areas. These nieas include the pioper 
tmpicineuialion ol quahly and manufacinring piwesses and ihe How down ofP-^S program 
requircnients lo supplteis. In sidditioa, alteiilioii lo dDcunieiitation niaiaieiiaacc ivxós to be a 
priarïty. • 

Oia' findings iuc allaclied lo Ihis iin;iiK)raiiduin foi yuiii rcvi;;w and coiiiini;iU. 
IJoD IJireclive 7650.3 iBquires Ifial reconwicndglions be i'esolved piraniptly. Please provide 
conmieiitü lliat state whcthcr yoii agrcc oi disagicc vvith tlic fiiidiiigs and lecomnicndations, tf 
yoii agicc wiih oui' recomniendaiions, dcscribe wbat actlons yon have taken or pliin lo lake lo 
Hccutnplisli the recuniineiidations iiiid iticliidc llie coinpldiuii diilts uf yoiir <ictioiis. If yuii 



' We condiictcd the nsscssment in sccordance vvilli Uic Council of the Inspector General on 
Itilegi'ity and Bt'ficicncy Quality SlitiidHrds for lns])cc1ioii and Hvaluaiion and giiiduiicc in 
ASyiOl, Qualily Managemeiil Syslems Aiidit Reqiiireiïients for Aviation, Spacc, and Defensc 
Organizations. 



and/ur itieuiiiplett:. 
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disagree wiüi Ihe recommeiMlfitions oï aiiy part of them, please give specillc reasons \vli.v yon 
ttisa(4ri}e and prapose alteiiialivc actiüii ifappioprialc, 

ir pos si bic, seiid n portable docimicnt tbrmal (,pdO Hlc contninina your commenis lo 

^^^^^mmm. C^pius oryour coinmciiLs must have Ihc acdial signaltire of the 
aiiiliuriziiiy ufllciHl füi yoiii' oi-jjiuiiailioii. We aie iiiitible to accept Ihe /Sigiiod/ syinbol til i)liice 
ofllie iitiltial sigiwüiic. It'yoii aiiange to scnd classilled cnmmenis Elecliniiically, ymi nutst seitd 
thuiTi (ïVL'i iftL- SnClilH' Itituniel Pn)lü(»l Router NL-twoi'k (S (PRNET). 

We slioitld feceive ymtr comtiietils hy Janiiarj' Ifi, 2013. Wi; iiHrnially iiitliidc copics of 
tlie comiueiHs in llie icpoil. Jl'you coiisidcr any niatteis to be cxeinpt tiom public release, yoii 
slintiid niai'k tliein cteai'ly tor Inspectoj' Oeneral r.diiRideration. 

Ouce we eunipkle oiu" assiessiiieuts, we will pi-ovide a i"epoi l on oiii' llHdiiigs, ttlong with 
atiy addilional imallers tlial migfil coine to our allcnlioii. Wc appicciatc the couiicsics cstciidüct 

10 




Ra I ld 1 pil R, Stone 

DcpiltY luspcctor Ociioial 
Policy urii:! Ovursigtil 



Altaclimcnt; 
As sialed 
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JOINT STRIKE FIGHTER PROGRAM 
ZOO 12™ Strcct South, Suite 600 
AriingtDn, Vlrgtnia 22202-5402 




From: 



Program Kxecutive OÖicer, F-35 Lightning II 



JAH 18 2013 



To: Deputy laspector General, Policy and Oversight 
Depm-tmcnt of Defense laspector Qeneral 

Subj: RESPONSE, NOTICE OF CONCERN - QUALITY ASSURANCE ASSESSMENT OF 
THE F-35 PROGRAM 

1. I roceived and analyzed your 19 Dcc 2012 Noticc of Conctfm-Quality Assurance Assessment 
of the F-35 Program (Project No. D2O12-DT0TAD-OO03) rcgaidiog yotir visit to United 
Technologies Corporation, Acrospace Systems (UTAS), performed at Ihe IntJependetiee, Ohio 
and Fort Worth, Texas facilities, 37 Novcmber-5 December 201 2. i appreciate your team's 
efforts to asscss cnnfonnity to quality management syslems, uonliaclual quality i;lauscs, 
and intemal processes and procedures, I view your assessment as a valuable tooi to 
independently evaluatt Ihc aruas for improvement witliin the F-35 Ptografn, with the ultimate 
goed of producing a quality, tiniely, and cost-cffective weapon systein for the United States and 
our allics, 1 take scrioualy my responaibility to ensure correclions are effeettvely implemented 
and die F-35 program ofliee, in fioncert with OUSD(AT&L), will föllow up at all identilïed 
locations to ensure compliancc with correctivc action plans. 

2. The F-3S Joinl Program Office (JPO), in concert with the Defense Contract Management 
Agency (DCMA), and Üie F-35 prime conlractor, liKkhced Martin Acrospace (LMA), re^ncwed 
cach of the a:iïe<is-ment's findin];s and recommendaiions. I accept all uf the tltidiiigs i&sued 
relative to UTAS and LMA. 1 agree that all of the recommendahoa'5 issued relative to UTAS 
and LMA ahouid bc asscsscd and impicmciitcd to the mfuiimum practicable etctent, The JPO 
acknowledges ihc findings issued sgainst DCMA Cleveland. and wiU collaborale with DCMA to 
ensure recommendutiuns ure assessed md implemented to the maximum practicable extent, A 
detailed disoussion of our path forward is contained at Appendix A and is markcd for Inspeelor 
Genera] Cunsideration. 

3. We will continue to work with you and your team during the quality assurance assessments of 
the remaining sclected F-35 suppliets. 




CHRISTOPHER C. BOGDAN 
Lieutenant General, U.S. AirPorcc 



Appendix A 



Copy to: 
OUSD(AT&:L) 
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JOINT STRIKE KIGHTER PROGRAM 
200 12"* Street South, Suite 600 
Arliniüton, Virgiiiia 22202-5402 




Director, Engineering 

JSF Weapon System Pragram Manager 

Director, Production 

Director, Program Inlegralion 
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JOEVr STHIKE FIGHTER PROGRAM 
200 12"' Streer South, Suil*! 600 
ArtingloD, VirgjnU 22202-5402 




FOR INSPECTOR GENERAL CONSIDERATION 



A ppendix A: 

1 . The F-35 Joint Program OfEci: (JPO), in aanctit with ihc Defenac Contract Management 
Agcncy (DCMA), and the F-35 prime confraclor, [jjctdieed Martiri Aerospace (LMA), reviewed 
eauh of the assessment's fïndings and reeummendations. I accept all of the fïndiiigs issued 
relative to TJTAS and LMA as vahd. I agree that all of the recommendations tssucd relative to 
UTAS and LMA should be assessed and implcmenled to the maximum practicable cxtent, 1 
believe that cvery finding has a deficiency that needs to be further addresscd by our program 
ofKce. However, we may discover duitng our root cause aiialysis and development of corrective 
action plans that a particular aspect of a finding may already bc satisfactortly addresscd and 
[ïxed, I lake ;jeriously my responsibility lo eusure correctiotis ate cffeciively iniplemertte<J aad 
the F'ï5 program otlice, in concert with OUSD(AT&L), will tbllow up at al! idcnlifiad locahons 
to cnsurc complioncc with corrective action plana. Further, I intend to ensure that ony 
corrections and improvements do not stay restricted to one location but are distiibutcd and 
implemented througltout the JSF enterprise. 

2. Regaiding the findings issued relative lo UTAS, tJie F-35 JPO lias estoblished a recurring 
joint working gronp amnngthe F-35 JPO, UTAS, I.MA, and DCMA to cnsurc all of the findings 
and reconunendations are understood and to start developing corrective action plans, Points of 
Contact wiüiin eacli of the four organizations have been establishcd for cvery finding <compkted 
15 Jan 1 3) and discussions have alieady commenccd. The F-35 JPO has begun working wilh 
ÜCMA at both IJl AS locations and at LMA Kort Wurth to undcrstand and properly charBCterize 
each finding with nn eye towards drafting Coirectivc Action Requests (CARs) for appropnate 
findings ton-güiog, ECD 1 5 Feb 13), The DCMA CAR proccss provides the Ooverrancnt a 
systeiriatic approach to ensure that corrective actions are taken in a timely manner The F-35 
JPO, in concert with the responsible DCMA organization, UTAS, and LMA, will implemcnt 
nionthly adjudication meetings witli each location ijicluded in tlie asseasment to ensure 
appropriate progress is hein g accnmplished in the corrective acüon plan for each finding, The 
first meeting with UTAS, LMA, and r>CMA will be held in March 2013, 

3. Regarding the findings issued reiative to LMA; they has been tolled into the tracking process 
initiated during the first visit to LMA, Fort Worth, and will be included in the LMA monlhly 
Iracking meetings. 

A. The findings relative to DCMA Cleveland are being actively atldressed DCMA Cleveland has 
bcgun a scrub of actiuns idcntifitd in the Letters of Dclcgation front DCMA Lockheed Martin, 
Fort Worth to ensure tliat all are being addressed. DfJMA C'leveland Is workmg wilh the 
cürttractor to iraprove the Qow of waivers/deviations, eiisuring adequate revlews and 
recommendations are provided to the delegating authnrity, Regarding the finding on 
rcdelegationof Material Review rcsponsibilities, DCMA Cleveland is assesstng all 
documentatlon to ensure that correct delegations are in place. These actions will hc complctcd 
by 15 April 2013 
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Appendix D 

F-35 Joint Program Office and Defense Contract 
IVIanagement Agency Comments 



Depuiy Iisspeetor Uciieial Poltcy and Oversiglit 
4800 Mark Centei- IJrive 
Alesaodria, Virfiinia 22350-1500 

Deai Mt, Slone, 

Tliank yow fov llte opportunity to eotninem on ymir ctrafl report, "Qiiality Assucaiice 
Assessmeirl of Uie Ligliling II Progiani", dalcd 5 August 2013. 

Both t1ie F-35 Joint Progiam Ofïïcc (JPO) and the Defciisc Conlracl Maiiagcincirt 
Agency (DCMA) lake veiy scriously Ilic findings and rcconimeiidalions yoii offer in yoiir iiiporl. 
Wc fippreclnic Ihc DoD IG's efforts to iiirprovc Itie F-35 Piogram imd foiitid Ihc DoD IG Team 
to bc bolli professional aiiri kiioivlcdgeable. As is notcd in Enciosiirc I , P-35 Jl'0 and DCMA 
Response tn Findings/lïecoinmendatioiis, wc have been nggressivcly addvessiiïg these fnidings 
since tlie bcginiiing of tiie assessment, Tire JPO and DCMA liavc initiatcd 343 Cori cctivc 
Aelion Plans {CAPs) to addrcss all 363 findings. As of 20 Augus[2013, 250 CAPs Por specific 
CQiilrticloi findings liave been validaictl nnJ closcd. ivilli 73 CAPs in vvoik, and 10 CAPs (o be 
oppi'oved, We ai o iniple inenting con-ective actions with Ihe ultiinate goal of prodncing a qnality, 
ttiiiely, and cost-eCfective weapon sysleni for tlio United States and ovr allics. 

Enelosnie 1 provides a dctaüed response Foi cach finding and lecommendation. 
Eiiclosiiie 2 pi-ovides recorrancndcd clianges lo veihiagc in tlic diafl leport. 

The action officers for this response are Mr. Tm Tmyors, lini.lraversrft'.isl'.niH [703) 501- 
5050 and Col Alex Stathopoulos, alcx.slalhoooiilo s@ dcina.mil . (SI 7) 763-4422, 




F-3S LIGHTIHING H JOINT PROGRAM OFFICE 
200 12"' Sdeet .South, Suite 600 
Ariinetnn, VUghila 21201-Sm 




Mr, Raiidolph R. vStoiie 



Sincciely, 




T.ïciifenant General, USAF 
Piogram Exccnlive Officcr 



Cliief Operaüons Officcr 

Defensa Conlracl Management Agcney 



Eiiclosiires: 

1. F-35 JPO and DCMA Response to Findings/Recoinmcndations 
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F-35 Joint Program Office and Defense Contract IVIanagement 
Agency Comments (cont'd) 



2. F-35 JPü commciits/iccomtnendcd changcs 



cc: 

OUSD(AT&.L) 

Asslstaiil Sc:ci'£[iii7 oTflicNiivy (Rcscarcli, Dcvelopciieiil & Acqiiisition) 
SAF AQ Principal Dcpiily 
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F-35 Joint Program Office and Defense Contract IVIanagement 
Agency Comments (cont'd) 



MEMORANDUM FOR DEPARTMENT OF DEFENSE, INSPECTOR GENERAL, AUDIT 
POLICY AND OVERSIGHT 

SUBJECT: Draft Report Quality Assurance Assessment of the F-35 Lightning II Program 
(Project No. D2012-DTOTAD-0003.000) dated August 5, 2013 

Reference: DoDIG Draft Report Project No. D2012-DTOTAD-0003 

We have attached the Headquarters, Defense Contract Management Agency' s comments 
and documentation as requested. 

The Point of contact for this our response is Col Stathopoulos at (817) 763-4422 or 
Alex.Stathopoulos@dcma.mil. 




DEFENSE CONTRACT MANAGEMENT AGENCY 

3901 A Avenue, Building 10500 
Fort Lee, VIRGINIA 23801 



August 29, 2013 




Attachment: 
As stated 
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F-35 Joint Program Office and Defense Contract Management 
Agency Comments (cont'd) 



Qualily Assurance Assessmenl ofllie F-35 Liglilning 11 Program 

Projett No. D201 2-DTOTAD-0003,OOo"' 
F-35 JPO and DCMA Findings, Recommendarions and Responses 

Finding A: The F-35 Joint Program Office (JPO) did nor ensure that Lockheed Martin and its 
subcontrattors were applying rigor to design, inanufactLiring, and cjjality assurance processes. 

K-SS JPO Response: Agree 

A rigoroiis design, manufactLiring, and qiiality assurance process exists The Program has 
COntinLied lo follow the DoD Systems Engineering process that spans from developmetit througli 
sustainment. Program plans sueh as the F-35 Syslems Engineering Program Plan, F-35 Program 
Manufacturing Plan, and F-35 Quaiity ManagemeJit Plan were writteii by the contractor, 
reviewed by JPO, and have been in place since early in the Systems Developinent and 
Demonstratioii (SDD) pliase and continue lo cvolve as the Program matiiics Program rcviews 
such as Critical Design Revicws, Prodiiclioii Rcadiness Rcviews, and various Quaiity Assurance 
reviews (for example First Arlicle Inspeclions) have been put in place with the F-35 Joint 
Program Office and DC?MA partidpation. Major reviews have detailed entrance and exil criteria 
to ensure requirements are deflned, inipleinented, traceable, and veritied by Program and 
Department of Defense Subject Matter Fxperts (SMEs) participating in the major reviews 
(providing both intemal and independent assessmems). 

The F-35 baseline design has continued to evolve as a resutt oT testing, producibiUty 
iinproveitients, and alTordability. Contractor and Ciovernment change boards are in use. Design 
change manageiTient has also evoKxd with the newly stüod up Coneurrency Integrated Product 
Team (IPT) which ensures that proper systems engineering, production, and sustainment reviews 
have been conductcd and the change can be iticoiporated to meet cost, Schedule, and 
perforaiance requirements. 

The Manufacturing plan has continued to evolve largely due to on-going concurrent design 
development and transition to rate production lines. Policies and procedures are in place to 
provide cheeks and balanees in upgrading the manufacturing operations, production floor layouts 
and looling, and procurcmcnt operations to cnsurc maituaitiing product iiitcgrity. 1-inditigs of 
independent audits by contraciors, DCMA, and DoD IG higbliglit opportunities for 
improvement. These audils and DCMA inspections have helped ensure greater discipline in 
adtiering to tbeir policies and procedures already esiablished. Addilionally, the Defense Contract 
Management Agency (DCMA) had issued a Level III Corrective Action Request (CAR) on July 
2010 for action to be taken in ensuring process discipline and procedural adherence. 

Recdmmendadon Al : (JPO) Ensure corapliance tO AS9100 thicughont the F-35 supply chain, 
F-35 JPO Response: Concur 

AS91 00 (Quaiity Management Systems - Requirements for Aviation, Space, and Defense 
Organirations) is an F-35 contractual requirement. The prime contractor (Lockheed Martin 
{LM)) is AS91 00 ccrtified and has flowcd down this quaiity rcquircmcm to its supply chain. 
AS91 00 certification is grantcd by a third party accrcditcd cnlity. Pcriodic rcviews are held by 
that third party entily lo maintain certification. Annual self audils are conducied by both the 
prime contractor and its suppliers to monitor compliance within their respective facilities. The 
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Government has had limited insight to results of these self audits, however, DCMA performs 
onsite process reviews as part of its oversiglit activity. DCMA at [,ocklieed Martin Fort Worth 
reorganizcd in l'cbruary 20 L"! to providc more focus and iiispcctioii cmphasis to cnsiirc AS9iOO 
compliancc and a quality product. Additïonally, JPO rcvicws AS91 00 ccrtificatioii status of 
critical supplicrs during tlicir antiual Production Rcadiiicss Reviews (PRR) "J licsc rcvicws serve 
as a LRIP risk ivianaiiemeiil tooi coiidueted with tlie prime coiitractor and selected major critical 
suppliers, coveritig Technical, Production and Sustainmeiit requirements. Suppliers must 
demonstrate consistent adherence of product compliant to coiitrattLial (AS*) 100) and applicable 
statutory and regulatory requirements. Findings of this IG audit have caused both contractor and 
Goverameiit eiuities to adjust the frequency and scope of independent audits to cnsine greater 
insight to AS9100 compliance. 

RecommendHtiun A.2: Ensure that Lockheed Martin approves all design and material review 
board changes and variances with Government conturrence. 

K-33 JPO Response: Concur 

Current F-35 Program plans (stich as the Configuration Management Plan) requtre that alt major 
design changes and variances for production aircrafl receive C3oveminent concurrence. Tlie 
Government maintains limited insight of tbe contractor's Material Review Board (MRB) 
process. Subsequently, the Government has reprioriti/red and refined its engagement in that 
process to ensure improvcd cffcctivcncss. l or nonconformances that rcquire MRB dispositioti, 
JPO dclegatcs the approval of Class II (minor) changes to DCMA and rcvicws all Class I 
changes thal affect form, fit, or function. A Class I change requsst is dispositioned tlirough the 
JPO C'hange Review Board (JCRB), which performs a leehnical and business assessmeiit and 
provides a written recoinmendation to the Joint Configuration Control Board (JCCB). With the 
input of major stakeholders, the JCCB adjudicates the recommended change and provides that 
decision to Contracts, who notifics the Prime Contractor to procecd iti iucorporating the chatigc. 

Any subcontractor submittal for variances and engineering change requests, for which Lockheed 
Martin is the sole design authority, mustbe go through Lockheed Martin's change control boards 
for review and approval. Furthermore, Lockheed Martin inust provide concurrence for 
engineering glianges relating to subcontractor owned designs, Various configuration 
management checks and balances (such as Physical and Functional Configuration Audits) are in 
place to ensure control. 

Recoinmeiidiitioii A.3: (JPO) Perform process proofing of all critical processes to include first 

article inspections, 

F-35 JPO Response: Partially Concur 

The Govcmment does not have the resources to perfoim process proofing of all critical 
processes. Responsibility and accountability rests on the prime contractor, with oversight from 
the Govemmeiil. Process pi'oofing and Firsl Ai'ticle Inspections are Program Plan and 
contractual requirements. The Government perfonns audits of process compliance to ensure the 

DJSrRIBUrrON state me mr A. Aftpfóvéd f^r publif rëtê^^ diitfibutiórl 1^ ui^lIrtlitÉCl, 

2 



116 I DODIG-2013-140 



Appendixes 



F-35 Joint Program Office and Defense Contract IVIanagement 
Agency Comments (cont'd) 



Quality Assurance Asisessmetit of the F-35 Lightning II Program 

Project No. D201 2-DTOTAD-OOO3.000 
F-35 JPO and DCMA Findings, Recommendations and Responses 

contractor 15 controlling critical operarions. The Government closely monitors strap, rework, 
and repair to ensure the contraetor is conducting proper root cause analyses to iinplement 
appropi iatc coricclivc actions. The Govemtnent also monitors the quality of delivered product to 
ensure c ontrol of esc ap cd defects. 

JPO requires the conlraclor 10 perförm First Article Itispections (FAl) and process proolmg as 
part of their implementalion of AS9100 and the F-35 Prograin Quality Management Plan. (Dver 
27,500 original baseline FAIs have been completed and approxiimately 400 more delta FAIs will 
be perfonned for parts due to eoncurrency, prodiicibility, process chaiiges, etc. In addition, the 
prime coturactor has ittiplcmcntcd varialioii maiiagcniciil, which is aii advaiiced quality systera 
tcchiiiquc tlial focuscs oti defect prcveiitioii and cominiious improveraciit tlirough the 
Identification of key product and process characteristics, This allows greater control of the 
manufacturing process. 



Recommendatioti A.4: (JPO) Modify lts comracts to iiiclude a quality escape clause, to ensure 
the Govenimcnt does iiot pay for nonconformiiig product. 

F-35 JPO Response: C'oncur 

JPO concurs that greater incentives should be taken to predude the cost of poor quality, Major 
non-tonfomiances that do not meet engineering specifïcation are identified and adjudicated for 
acceptaiicc or rejectioii witli considcratioiis,''witliholds placcd on a variance. Minor noii- 
conformances are reworked, repaircd, or uscd as is. I lie p^o}^raln has transitioncd to fixed price 
type contacts willi a 0/100 share ratio and claiises will prescribe 10 applicable FAR requiremenls. 
Along with considerations ..■ vvithholds 011 non-cotilonnanees, this contract siruclure will facilitate 
greater incentive by the contrattorto provide quality assurance in order to maintain cost and 
Schedule obligations without over running their negotiated budget. The contractor will absorb a 
levcl of cost for poor quality. In addition, the JPO continucs 10 esiablish quality performance 
targets witli commitmcnts from JPO and LM senior leadership. I hesc targets are based upon 
conlinuous learning in fabrication and asseinbly opetalions, correclive aclion impletrienialion, in 
addition to product and process improvements as a result of affordability and coneurrency 
changes. 



Recommendatioii A,5: (JPO) Assess the impacts and risks to all delivered aircrafl for all 
findings. 

F-3S JPO Response; Concur 

The impacts and risks of all findings were assessed as part of the Conective Action Request 
(CAR)/CorTective Action Plan (CAP) process. VVhen a CAR was written, the F-.'55 fleet was 
assessed for impact. Concurrcticc on eacli CAP considcred cffectivity, safety, and contract 
elements 1 his dctennined priority and timing required to inipleinetit the corrective action into 
the aftected aircraft as appropriate. 
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Findiiig B: JPG did nol flow down crilical safety item (CSl) requirements. 
F-35 JPO Response: Agree 

Public I.aw section S02, requiring a Critical Safety Item program was introduced in 

fiscal 2004 Icgislation, thrce ycars after the F-35 SDD contract award. Furtlier, guidaiicc frora 
DoD 011 how to impicmetit CSl was not immcdiatcly fonlicoming. As sucli, tlic program 
deiayed CSl impleinentation pending clear poliey and lo minimize unnecessary cost grovvth. As 
lliÈ name implies, the focus of TSI is. avialian safety. The F-35 program and coniraclor team 
treat safety as a Number ! priority. The JPO's developmem specification inckides stressing 
criteria foraviation safety (max loss of aircraft rate), ha:^ard risk levels, and mission completion 
siicccss criteria {'urther, LM's cnsjiiteci ing Standard work requircs a comprclieiisivc product 
integrily program to eiisurc the safety atid pcrformatice of safety / inission critical componcnts, 
subsysteins, and assetnblies. Rigorous guidelines to include redjndaiicy, backup systerns, design 
and quatification staiidards, etc. are itnposed on these components to ensure safety and 
performance. Many of the prescribed CSl practice, stich as 100% inspection, are subsumed 
under the umbrelia of the integrily programs; fracture critical; safety critical; diirability critical; 
and missioii critical. Noticthclcss, JPO and the contractor dcvciopcd and rcgiilarly update a list 
of CSl parts and asscmblies "f his list has been rcquircd on cach LRIP contract to date "I hal 
said, the JPO is phasing in the remaining tenams of a CS] program ihrough ongoing LRIP 
contracting actions and new SDD actiotis. For exainple, recent closure on the LRIP 6 contract 
includes additioual CSl related tasks for variatice iiiaiiageinent and notitkation, 



Reeommendiitioii B.1: (JPO) linplcmcnt an aviation critical safety item program that irieets the 
requirements ol the Public Law and DoD Policy, which would include flow down of 
requirements for a critical safety item program to Lockheed Martin and its subcontractors. 

F-3S JPO Response: Concur 

JPO is working with the prime coniraclor itt dcvclopitig a Reqiiest for Proposal (RfP) for a two- 
phascd CSl Non Rccurring Engineering (NRL) approach 1 his will rcquiie the dcvelopmcnt of 
comprehensive C"SI requirements which will be added to the SDD cotitract's Statement of Work. 
These requirements will address the identification of crilical characteristics that enable 
complianee at the prime contractor and throughoiit its siipply base. Once the SDD NRF, efforts 
are completed, the fuU scope of the F-35 CSl program will be included in futtire produttion and 
sustainment contracts. Commcnccmcnt of ccnaiu initial rcctining tasks will bc implcmcntcd 
using tliE LRIP 6 contract. In the interim, the contiactors' integrity and quality programs are 
providing asswance for continued aviation safety. 

Recommcndniion B,2: (JPO) Assess the impacts and risks to all delivered aircraft for critical 
safety item deflciencies. 
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F-35 JPO Response: Concur 

Deficiënties and impacts to critica! parts are identified and assessed during quality assurance 
practiccs and tcsting opcialioiis as part of tlic SDD dcsijiii and product vcrification processes. If 
safety issues and limitations are discovcrcd, cliatigcs are itiitïatcd and iticoiporatcd as warranted 
tothe test and prodirction aircraft. Itiitial and comitiucd aiiAvorthiness is a major focus of tlie I'- 
35 Class Desk, Airworthitiess Team, Integrated Test Team, and Inlegrated Product Teams. The 
SDD tesl aircrafi have more llian 5,000 llighl liours. The coiiibined F-35 fleel has accumulaled 
more than 7,000 flight hours. This is not to say that there are no significant issues Or discoveries. 
Rather, there is an upfront understanding whether any new diseovery poses an increased safety 
risk. If iicccssary, proper actions are taken, such as suspcnding flight opcratioiis, directing 
inspections, addiiig flight limitations or restrictions, and dirccting hardware or software 
modifïcations to mitigate the safety risks. 



Finding C: JPO did not ensure that Lockheed Martin llowed down quality assurance and 
technical retjuirements to subcontractors. 

F-35 JPO Response: Partially Agree 

Processes are in place, whicti ensurc that requirements are flowed down from the prime 
cotitractor to its supply base. But, there are areas for improvemenl. A significant iiuinber of IG 
findings were noted in this area. The four key areas of liigliesl concern were Firsl Article 
Inspection, Configuration Control, Design and Development Requirements, and Diminishiiig 
Manufacturing Supply and Materie! Shortages. 

First Aniclc Inspection (I-AI) - Ahhough the IG finding statcs that there is a lack ofdefmitive 
FAI requireinent flowed down to ihe prime contractor's supply chain, all major suppliers are 
required to adhere lo AS9I00 in which FAI is addressed. AS9102, Firsl Article hispeclion, is 
used as a guideline to assist in meeting the AS9 1 00 requirement. Vahdation of interface control 
features, key characteristics, interchangeability / replaceability, outer mold-line, coating 
application, and component end item inspections are coiiductcd throughoiit major asscmblics and 
tied to the aircraft acccptancc (DD25Ü) proccss As such, LM does not rcquirc FAIs to bc 
performed on major component assemblies such as the wing and the center, forward, and aft 
fuselages. 

Configuration Control - The associated IG finding states that there is a lack of requirement flow 
down and cominunication bctwecn Lockheed Martin and its suppliers, leading to configuration 
inanageraent issues. I hc I--35 maintains a detailcd Configuration Management Plan which 
describes configuration control through the prime contractor and its supply chain. C'heck and 
balanees have been put in place to ensure configuration control These ehecks and balances 
include (but are not limited to) Functional/Allocated/Product Baseline Reviews, First Article 
Inspections, Physical Configuration Reviews / Audits, Function Configuration Audits, and 
System Verification Reviews. Preparalion for these major reviews serves as a means to 
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conimunicate and understand ttie reqiiirements between the prime contractor and its suppliers. 
Execution of such reviews validates compliance and ensures configuration control. 

Design and Dcvclopmcnt Rcquircmcnts - A key flnding iii this arca dealt witli a lack of flow 
down of Missioti Systems Software Devclopraeiit rcquircmcnts, in additioii to vcrification and 
traceabiHly of ihose requirements. ll is true tiial tiiere are chaltenj|es with iraceabilily of mission 
systems requirements. However, it is nol accurate to say ihat the requirements are untraceable. 
Traceability can be accomplished. The diallenge is that the traceability requires access to 
several tools and is more manual in nature andnot an automatic Dynamic Object Onented 
Requirements Systciti (DOORs) functioii. fhc fiiiding also mcntioned the lack of maiiitcnaiice 
of the Air System ülock Plan. This statement is inaccurate. At soine poitit iti time this statement 
may have been irue, however, the Air Systein Block Plan is tnaintained via a C'ontract Data 
Requirements List (C'DRL) ihal is updaied every 6 months and the Program now has a Build 
Review Board that actively maiiages the capabihty content of the detailed build plans that 
constitLite the higher level Block Plans. The Build Review Board monitors execution progress 
and plaiiniiiji of ticw software builds to ensurc that capabilities are dclivcrcd on time based on a 
realistic schcdule and acts as the Decision Body that makes infonncd trades to support software 
execution and LRiP production deliveries, 

Diminishing Manufactunng Supply and Materiel Shortages (DMSMS) - This findiny stated that 
JPO had nol funded Lockheed Martin (LM) to implement a DMSMS program. The conversion 
from fundinji DMSMS on the SDD contract to the LRIP coiitracts was taking place when the IC 
was briefed. Diiriiig this interim. DMSMS was not funded by the 1-35 Joint Prograin OtTicc. 
Potemial risks were initigaied as a result of the priine contractor executing on intemal company 
funding and converting to an [Jndefitiitized Contract Aulhorization (IJC?A), with Full 
Autbonzation funded under LRIP 6. In addition, the DMSMS strategy was under review, the 
charter was signed, and the management plan was in draft by the F-^^i Joint Program Office. 
The prime contractor also rcviscd the Stipply Chain Management Plan which now incliidcs the 
DMSMS ■] cam as a stakcholdcr, ensuring proper coordination. I'utthermore the DMSMS 
Supplier Statement of Work requirement was updated to specifically require LM suppliers to 
flow down DMSMS requirements to sub-lier suppliers. Funding and improvements conceming 
DMSMS requirements have allowed better proaetive parts obsolescence management. 

Reconimeiidation C: (JPO) Pcrform tcchnical and quality assurance requirement flow down 
and verification throughout tlic i'-35 supply chain. 

F-35 JPO Response: Partially Concur 

The Government does not havé the responsibility Or resources tO perform requirement flow dowti 
venfication throughout tbe prime tontractor's entire supply chain. The prime contractor is 
responsible and accountable, with oversight from the Government. The Government perfonns 
spot chccks and conducts surveillance to ensurc the contractor is flowing down requirements to 
its subcontractors. J he Government monitors this closely through participation in Design 
Reviews, Production Readiness Reviews, Structural Equivalency Reviews, Program 
Management Reviews, production floor inspections, configuration audits, etc. On site 
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suneillance is primarily perfonned by DCMA to ensure the contractor and its subcontractors are 
conipiying with contractual requirements. 



Finding D: JPO did not establish an effective quality assurance organization. 
F-35 JPO Kesgionse: Agrec 

The F-35 Program management team is committed to establishing an effective Quality 
Assurance program and orgauization. It has empowered its Quality Team, under the Pi'oduction 
IPT, to define measurable Program quality requirements and is engaged in creating a program 
environment tbat fosters continuous iimprovemem. JPO's Quality Team is engaged during 
contract and product dcvciopmcnl to ensure quality considcratioiis are considercd. Their 
engagement coiuiiiucs throiigh fabrication and asscmbly to ciisiirc quality control and cominuous 
improvemenl llirough aircraft delivery. JPO lias itilegrated DCMA as part ofits Quality Team, 
per tlie long standing and ongoing Program IPT structure and the Prograin's Memorandum of 
Agreement (MOA) with DCMA, to enhance quality surveillance and performance monitoring. 
DCMA sen'es as an on-site agent and is responsible tor contract administration and has 
resources, training, and processes to cfiectivcly admitüster quality assurance. I hcir insiglit and 
oversight of contractor pracliccs placcs thcm in the best positioii to idciilify, report, and address 
quality assurance issues. Based upon some of the IG findings, JPO and DCMA are in the 
process ol'refuiing ilieir teaming structure to ensure more cogent focus on critical processes and 
product quality results. 

JPO has formcd a partnership with Lockheed Martin and DCMA. Each entity maintaïns its 

independent pcrspcctivc, biit is teamcd to acliicvc a coinmon goal - ensure the delivery of a 
quality product lo IJS Services and International Partners' warfigluers. It is through this 
arrangement tliat requirements are arliculated, executed, verified, and controUed. The 
Government (JPO and DCMA) continues to work closely with the contractor's Program and 
Core Quality organi^ations to ensure adequate support is provided to ineet Program objectives. 
As the Program transitions to Full Ratc Production, JPO acknowledgcs the nccd for a focused 
quality orgatiization. 



Recommendation D: (JPO) Establish an independent quality assurance organization, which has 
the authority and resources to enforce the AS51Ü0 Standard and F-35 produet quality, 

F-35 JPO Response: Non-concur 

JPO acknowledges opportunities to iraprove management of Quality performance and the 
benefits of realignment withiii the F-3.'i Program organizational structure to better support 
transition from Low Rate Initial Production to Full Rate Production. These efforts have 
comraericed lo include an increase in resources and realignment of the Quality Team to report to 
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program leadership. The JPO non-conciirs witli establisliiiig aii indepeiideiK quality assurance 
Organi^ation. DCMA perfonns the role of the independent quality assurance organii^ation tbr the 
F-35 and other DoD programs. DCMA conducts proccss surveillance agaitist the AS9100 
Standard and cnforces i'-35 product quality rcquircmeiits. Additioiially, the JPO Quality Team is 
stafïcd fiom two independent organizations; Air Force Life Cycle Management Center 
(AFLCMC) and Naval Air Systems Coimnand (NAVAIR). While serving on the JPO Quality 
Team, these individuals retain iheir responsibility to invoke their independent AFLMC and 
NAVAIR leadership in the event the JPO does not properly address quality concerns. 



Finding E: DCMA was not perfonning quality assurance oversight comraeiisurate with product 

criticalily. Iiisiifficicnt writtcn dircclion from the JPO coupled with inadequate 
executioii froin DCMA resulted in ineflfcctive Government oversight of the F-35 
program. 

F-3S JPO Response: Agree 

DCMA is an independent DoD component Ihat works ditectly with Defense suppliers and 
buying agencies to ensure contract compliance. DCMA partnership with the JPO is cmcial to 
delivery of the F-35. DCMA's oversight activities are focused on process surveillance to ensure 
systemtc issues are adequately addressed. Nonconformances are assessed according to severity 
and reeurrenge, Correttiie attion requests have been generated to address nonconfonnantes, in 
concert with communicating with JPO on the implementation of corrective attions. In addition, 
DCMA is an iiitcgral pait of the Material Review Board (MRB) process. J'hcy will continue to 
revicw the classification of all iionconformance, raising issues which affect form, fit, or function 
to JPO attention, and providing disposition to minor nonconfonnance as required. DCMA also 
ensures that the prime coniractor and ils suppliers are adequately employiny the MRB process. 
Contractor performance status is also provided monthly to JPO in functionally-aligned written 
reports. 

JPO feedback is critical to DCMA. DCMA also relies on its Headquarter reviews to nieasure 
performance based upon agency policy, contract content, and customer expectations JPO and 
DCMA have initiated improveiments in tlieir teaming relationship to include emphasizing 
priorities and exploring the sharing of resources. DCMA has been conducting suri'eillance 
activities throughout the supply thain sinte the beginning of the F-35 SDD contract. 137 Letters 
of Delegation (LODs) are in place with all major and critical paits suppliers and being 
contimially updated to meet program necds Additionally, as the Critical Safety Item (CSI) 
program matures, DCMA and JP(J will continue refocusing limited assels to ensure coverage. 
DCMA (Ft Worth) and its delegated team will continue to utilize the Corrective Action Report 
(CAR) methodology to focus contractor efforts and resolve risks whilê enhancing 
CARy'Corrective Action Plan conimjnication with the JPO. 
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Rccommcndalion E.la (JPO): Revise the Defense Contract Management Ayency 
memorandum of agreement to provide explicit surveillance criteria 

F-35 JPO Response: Coticur 

The N'Ieniorandum of Agrccmctit (MOA), annex A-P, datcd Fcbniary 201 1, between the JPO and 
DCMA is beiiii; updaled to reflect Program objectives requiring DCMA support in the following 
areas: Management of contractor Quafily Process implemenlalion, control, malui'ity, and 
verification; Materinl Review/Cliange Board Process authority and rdated task requirements, 
Foreign Object Damage (FOD) management, Governnnent Flight Representative surveillance, 
Safety of l-light inspectioii requirements, contractual Corrcctive Actlon Requesl iiotification. Air 
sysicm acccpiancc (DD250), Supply Cliaiii Management, etc. Ilowevcr spccific surveillance 
activities and priorities are aiticulated via other tools, such as Quality Assurance Letters of 
Inspectiën (QALFs), 

It is important to note that the MOA update, though crucial in sfiaping organizational roles and 
responsibilitics lliat addrcss llic findings in tliis report, is only pan of the correclivc aciion 
process bcing implcmcnted Exccution of DCMA policics, cnlianccd communication between 
the JPO and DCMA, and enhanced inter-DCMA Communications witb supplier DCMA 
organizations all play a i'ole in iniproving surveillance. In addition to the MOA, DCMA utilizes 
risk based sun-eillance plans to support F-.15 contraets. As part of the fimctional surveillance 
planning process, DCMA determines what tjpes of surveillance techniques will be applied to 
ensurc a supplicr has adequate quality processes in place and incet all contractual requirements. 
Risks are identificd through bistorical knowicdge of the suppliers systems and processes, results 
of surveillancÈ execution, analysis of DCMA inteitial data, supplier data and customer data. 

Rccanimcndiiiion E.l.b: {JPO) Ensure that Defense Contract Management Agency is 
perfotming quality assurance oversight commensurate with product criticality. 

F-35 JPO Response: Concur 

DCMA has been, and coiilinues to be, responsive to product criticality while performing its 
contact adminisiration quality assurance funclions. This includes optimizing quality assurance 
oversight of contractors' critital processes, operations, and product. Product criticality is 
govemed by Federal Acquisition Regulation derived surveillance strategies. FAR./DFAR 
tegulatory requirements as manifest in 1--35 contracts are also guidcd by DCMA policics. 
DCMA policics dcfinc risk considcrations and thresholds in dcvcloping surveillance plans. 
Qualily specialists determine \\4iat types of surveillance techniques wlU be applied to ensure a 
supplier has adequate quality processes in place. Surveillance activities and data analysis allow 
DCMA to re-evaluate risk levels and where appropriate adjust surveillance plans. 

As non-confonnance is identified, JPO may request special surv eillance be perfonned in other 
areas duc to scvcrily or whcihcr lliey are systematic in nature, "fhis is captiiicd in other 
docunients such as OALl's. "I hc DCMA momlily report also is a means of communicating 
quality assurance oversight as it pertains to production status. Continuous communication 
between JPO and DCMA to discuss and resotve quality issues is critical to meeting Program 
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Appendixes 



F-35 Joint Program Office and Defense Contract IVIanagement 
Agency Comments (cont'd) 



Quality Assurance Assessment of the F-35 Lighming II Program 

Project No. D201 2-DTOTAD-000:vOOo"' 
F-35 JPO and DCMA Findings, Recommendations and Responses 

objectives. Perfomirg quality assurance in accordance with product criticality is central to how 
DCMA conducts contract surveillance. 

Reconimendatioii E.2,a: (DCMA) Provide a coraprchctisivc quality assurance ovcrsight plan 
for Jfoint Program Office approval to be incliided iti the inemorandura of agjeenieiit. 

F-35 Dt'MA Response: Partially Conciu' 

DCMA will eiisLire tlie MOA, as Currently being updated, corrtplements DCMA risk basêd 
surveillance planning. The DMCA surveillance planning process is nsi< based and developed 
throiisjh historical knowlcdgc of lire supplicrs systems and processcs, rcsults of surveillance 
execiition, atialysis of DCMA iiitcrnal data, siipplicr data and CListoracrdaia. 

DC^MA is ako implementing a dedicated Supply Chain Management team at LMFW to address 
the significant supply chain oversight challenge of the F-35 program, The focus of the team vvill 
be to evaluate LMFWs supply chain processes and system as well as a more focused emphasis 
on risk suppliers throiighout the Supply Chaiii. 

Reconiinendalioir E.21): (DCMA) Audit the execution of the quality assurance oversight plan 
thtoughout the F-35 supply chain. 

F-35 DCMA Response: Concur 

DCMA will coinply with its quality assurance oversight plan, Regular audits of supplier atid 
prime DCMA organizations are crucial to inaximizing the effectiveness of risk based 
suf%eillance Within DCMA, Management Review Teams (MRT) provide comprehensive 
functional reviews on our quality program. These risk-based audil eflbrts, are coiiducted by 
independent quality experts and occur throughout the entire DCMA organi7atiün ensuring 
coverage of the ¥-^5 supply chain. Continued communication of audit results with our 
custoniers and enhanced coinmuiiication betweeti DCMA supplier CMOs will also suppletnent a 
robust audit process. 
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Acronyms and Abbreviations 



Acronyms and Abbreviations 



ANSI 


American National Standards Institute 


AS 


Aerospace Standard 


ATP 


Acceptance Test Procedure 


BOS 


Back-Up Oxygen System 


CAR 


Corrective Action Request 


CDR 


Critical Design Review 


CSI 


Critical Safety Item 


CTOL 


Conventional Takeoff and Landing 


CV 


Carrler-Suitable Variant 


DCMA 


Defense Contract Management Agency 


DMSMS 


Diminishing Manufacturing Sources and Material Shortages 


DOORS 


Dynamic Object Oriented Requirements System 


DU 


Display Unit 


EMI 


Electromagnetic Interference 


ESD 


Electrostatic Discharge 


EU 


Electronics Unit 


FAI 


First Article Inspection 


FMECA 


Fallure Modes, Effects, and Criticallty Analysis 


FOD 


Foreign Object Debrls 


FR AC AS 


Fallure Reporting, Analysis, and Corrective Action System 


ICD 


Interface Control Document 


lEC 


International Electrotechnical Commission 


IPT 


Integrated Product Team 


ISO 


International Organization for Standardization 


JACG 


Joint Aeronautical Commanders Group 


JPO 


Joint Program Office 


JSF 


Joint Strike Fighter 


LOD 


Letter of Delegation 


LRIP 


Low-Rate Initial Production 


LSLM 


Limited Shelf-Life Materials 


NOC 


Notice of Concern 


OBOGS 


On-Board Oxygen Generation System 


OFI 


Opportunity for Improvement 


OIG 


Office of Inspector General 


PCD 


Panoramic Cockpit Display 


PDR 


Preliminary Design Review 
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Acronyms and Abbreviations 



QN Quality Notifications 

SDD Software Design Document 

SDP Software Development Plan 

SOW Statement of Work 

SRS Software Requirements Specification 

STOVL Short Takeoff and Vertical Landing 

SVDD Software Version Description 

UTAS United Teclinology Aerospace Systems 

VDD Version Description Document 
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Whistleblower Protection 

U.S. Department of Defense 

The Whistleblower Protection Enhancement Act of 2012 requires 
the Inspector General to designate a Whistleblower Protection 
Ombudsman to educate agency employees about prohibitions on 
retaliation, and rights and remedies against retaliation for protected 
disclosures. The designated ombudsman is the DoD IG Director for 
Whistleblowing & Transparency. For more Information onyour rights 
and remedies against retaliation, go to the Whistleblower webpage at 
www.dodig.mil/programs/whistleblower. 



For more information about DoD IG 
reports or activities, please contact us: 

Congressional Liaison 

Congressional@dodig.mil; 703.604.8324 

DoD Hotline 

800.424.9098 

Media Contact 

Public.Affairs@dodig.mil; 703.604.8324 

Monthly Update 

dodigconnect-request@listserve.com 

Reports Mailing List 

dodig_report-request@listserve.com 

Twitter 

twitter.com/DoD_IG 



